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AIR  FORCE  OPERATIONAL  TEST  AND  EVALUATION  CENTER 
KIRTLAND  AIR  FORCE  BASE,  NEW  MEXICO  87117-7001 


EXECUTIVE  SUMMARY 


1.  An  integrated,  tridepartmental  (Department  of  Commerce  (DOC),  Department  of 
Defense  (DOD),  and  Department  of  Transportation  (DOT))  test  team  under  the  overall 
management  of  the  Air  Force  Operational  Test  and  Evaluation  Center  (AFOTEC)  conducted 
the  Initial  Operational  Test  and  Evaluation  Phase  II  (IOT&E(2))  of  the  Next  Generation 
Weather  Radar  (NEXRAD)  system.  The  test  team  conducted  IOT&E(2)  between  6  March 
and  6  August  1989  at  three  test  sites:  Norman,  Oklahoma;  Tinker  Air  Force  Base  (AFB), 
Oklahoma;  and  Oklahoma  City,  Oklahoma.  The  purpose  of  IOT&E  (2)  was  fourfold:  to 
evaluate  the  operational  effectiveness  and  suitability  of  the  preprcduction  NEXRAD  for 
DOC,  DOD,  and  DOT  to  support  a  full-production  decision;  to  review  deficiencies  and 
enhancements  documented  by  the  test  team  during  IOT&E(1A)  and  IOT&E(1B);  to  identify 
deficiencies  and  enhancements  not  previously  documented;  and  to  identify  items  to  be 
addressed  during  follow-on  operational  test  and  evaluation  (FOT&E). 

2.  The  NEXRAD  system  is  a  major  upgrade  of  existing  weather  radar  capabilities  to 
support  the  weather- related  missions  of  the  DOC,  DOD,  and  DOT.  The  NEXRAD  system 
is  designed  to  use  Doppler  radars  to  obtain  storm  intensity  and  quantitative  information 
on  wind  structure  within  storms.  During  IOT&E(2),  a  single  validation  phase  preproduction 
NEXRAD  system,  consisting  of  one  Radar  Data  Acquisition  (RDA)  unit,  one  Radar  Product 
Generation  (RPG)  unit,  and  four  Principal  User  Processing  (PUP)  units,  was  tested.  The 
RDA  unit  included  a  Doppler  radar  and  the  software  required  to  perform  signal  processing, 
clutter  suppression,  control,  error  detection,  and  calibration.  The  RPG  unit  included  all  the 
hardware  and  software  required  for  real-time  generation,  storage,  and  distribution  of  radar 
products  for  operational  use  and  for  overall  NEXRAD  system  control,  status  monitoring, 
error  detection,  and  data  archiving.  The  PUP  unit  included  all  the  hardware  and  software 
for  the  request,  display,  storage,  and  annotation  of  products.  It  also  included  the  hardware 
and  software  for  the  control  of  the  PUP,  status  monitoring,  and  data  archiving. 

3.  This  test,  the  second  phase  of  the  NEXRAD  operational  testing  (IOT&E(2)),  was 
conducted  in  two  parts.  Part  A  was  a  shared  development  test  and  evaluation  (DT&E) 
and  operational  test  and  evaluation  (OT&E)  period.  Part  B  was  dedicated  to  OT&E. 
IOT&E(2)  was  based  on  18  objectives  (8  effectiveness,  7  suitability,  and  3  combined)  as 
identified  in  the  approved  IOT&E(2)  test  plan. 

4.  The  evaluation  of  the  operational  effectiveness  objectives  relied  on  the  opinion  of  the 
NEXRAD  operators  obtained  through  questionnaires.  During  the  Operator  Questionnaire 
administration,  the  operators  provided  two  responses  for  each  question.  The  first  response, 
discussed  in  paragraphs  5a  and  7  below,  evaluated  the  system  when  it  was  operating 
disregarding  system  outages  and  was  used  to  evaluate  NEXRAD’s  effectiveness  in 
comparison  to  the  users’  criteria.  The  second  response,  discussed  in  paragraph  5d. 
evaluated  overall  system  performance  including  the  impact  of  system  outages  and  is 
provided  as  additional  information. 

a.  Operations.  NEXRAD  met  the  operators’  minimum  operational  requirements  as 
an  aid  in  providing  weather  warnings,  advisories,  and  routine  weather  services  support, 
primarily  because  of  the  accuracy  and  high  resolution  of  reflectivity-based  products.  The 
capability  to  magnify  and  time-lapse  storms  in  a  color  presentation  was  particularly 
effective.  However,  several  deficiencies  were  identified.  During  widespread  convective 
weather,  the  usefulness  of  the  velocity-based  products  was  severely  degraded  because 
they  contained  large  areas  of  range-folded  and  incorrectly  dealiased  data.  In  addition,  the 
operators  did  not  find  useful  information  in  the  layered-turbuience  products.  Use  of  the 
Unit  Control  Position  and  the  PUP  together  resulted  in  a  significant  increase  in  operator 
workload.  During  severe  weather  situations,  the  radar  often  failed  to  recover  automatically 
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from  power  transitions.  In  addition,  test  team  specialists  identified  numerous  deficiencies 
with  planned  agency  operations  training. 

b.  Logistics.  The  NEXRAD  system  did  not  meet  the  users’  requirements  for 
maintainability,  fault  isolation,  and  availability.  In  addition,  the  test  team  identified  reliability 
problems  with  the  preproduction  transmitter,  RPG,  the  graphics  processors,  and  the  optical 
disk  drive  units.  Agency  technicians  were  not  able  to  maintain  the  system  within  the 
required  repair  time  with  the  technical  manuals,  training,  and  primary  fault  isolation 
capability  provided  for  IOT&E(2).  The  Preliminary  Technical  Manual  set  was  incomplete 
and  contained  numerous  errors  making  it  inadequate  for  training  and  for  maintaining  the 
NEXRAD  system.  Training  did  not  contain  sufficient  detail,  did  not  interrelate  functionality, 
and  did  not  follow  a  logical  plan;  therefore,  technicians  did  not  develop  the  required  skills 
to  maintain  NEXRAD. 

c.  Software.  The  documentation  and  source  listings  for  the  four  computer  program 
configuration  items  evaluated  met  the  users'  requirements.  Software  evaluators  found 
individual  source  listings  contained  simple,  expandable,  modular  code  characteristics. 
However,  problems  were  identified  with  the  overall  system  software  documentation. 
Software  personnel  were  often  unable  to  find  or  trace  required  information.  Software 
training  did  not  provide  the  required  skills  and  procedures  for  software  maintenance. 
Detailed  agency  plans  for  project  and  configuration  management  were  incomplete  an d  had 
not  been  finalized  and  approved.  This  may  impact  the  government’s  ability  to  assume 
software  support  responsibilities  at  the  appropriate  time. 

d.  Overall  Performance.  When  the  overall  performance  of  NEXRAD  was  considered, 
including  the  impact  of  system  outages,  the  median  questionnaire  response  of  all  the 
operators  indicated  that  the  system  did  not  meet  their  requirements  as  an  aid  for  providing 
weather  warnings,  weatner  advisories,  and  routine  weather  services.  Most  operators  stated 
that  NEXRAD  was  often  not  available  to  support  these  services  because  of  PUP  lockups, 
system  outages,  and  problems  with  recovering  automatically  from  power  transitions. 
However,  possibly  because  of  their  smaller  area  of  weather  support  responsibilities.  DOD 
median  questionnaire  responses  indicated  that  the  system  met  their  minimum  operational 
needs  when  the  overall  NEXRAD  performance  was  considered. 

5.  The  test  team  reported  deficiencies  and  enhancements  in  accordance  with  Air  Force 
Technical  Order  00-35D-54.  During  the  NEXRAD  IOT&E(2),  the  test  team  validated  and 
submitted  545  new  service  reports  (SRs)  to  the  Joint  System  Program  Office:  486  were 
deficiencies  and  59  were  enhancements.  Additionally,  during  this  period  the  test  team 
revalidated  87  deficiencies  and  23  enhancements  from  the  355  SRs  submitted  during 
previously  conducted  lOT&Es.  With  regard  to  safety  deficiencies,  the  test  team  identified 
56  safety  deficiencies,  9  of  which  were  potentially  life  threatening  or  could  cause  severe 
injury  or  occupational  illness. 

6.  The  test  team  identified  several  items  which  should  be  addressed  during  follow-on 
operational  test  and  evaluation  (FOT&E).  The  three  most  significant  items  are  described 
below.  For  operations,  because  of  limitations  identified  in  paragraph  2.2.2a.  FOT&E  should 
address  the  responsiveness  of  a  production-model  NEXRAD  in  an  operational,  multiple- 
user  environment  during  a  significant  weather  season.  For  logisfics.  organizational-level 
maintenance  should  be  performed  on  a  production-model  NEXRAD  using  validated  and 
verified  technical  manuals  and  the  integrated  logistics  support  infrastructure.  For  software, 
the  operational  support  facility  should  generate  and  test  a  new  software  version  release 
well  in  advance  of  support  management  responsibility  transfer  (SMRT)  to  include  adding, 
deleting,  and  changing  functionality  within  the  RDA.  RPG,  and  PUP. 
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7.  In  summary,  NEXRAD  was  an  effective  aid  in  providing  weather  warning,  weather 
advisory,  and  routine  weather  services  support.  However,  the  usefulness  of  velocity- 
based  products  was  severely  degraded  during  periods  of  widespread  convective  activity. 
In  addition,  reliability,  maintainability,  and  availability  problems,  such  as  those  associated 
with  power  transitions,  PUP  graphics  processors,  and  the  transmitter,  severely  detracted 
from  mission  capability.  Software  documentation  and  software  support  resource  deficiencies 
produce  a  risk  that  the  government  may  not  be  able  to  assume  software  support 
responsibilities  at  the  appropriate  time. 
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SECTION  I  -  PURPOSE  AND  BACKGROUND 


1.0  OPERATIONAL  TEST  AND  EVALUATION  (OT&E)  PURPOSE.  An  integrated, 
tridepartmental  (Department  of  Commerce  (DOC),  Department  of  Defense  (DOD),  and 
Department  of  Transportation  (DOT))  test  team  under  the  overall  management  of  the  Air 
Force  Operational  Test  and  Evaluation  Center  (AFOTEC)  conducted  the  Initial  Operational 
Test  and  Evaluation  Phase  II  (IOT&E(2))  of  the  Next  Generation  Weather  Radar  (NEXRAD) 
system.  The  test  team  conducted  IOT&E(2)  between  6  March  and  6  August  1989  at  three 
test  sites:  Norman,  Oklahoma;  Tinker  Air  Force  Base  (AFB),  Oklahoma;  and  Oklahoma 
City,  Oklahoma  The  purpose  of  IOT&E  (2)  was  fourfold:  to  evaluate  the  operational 
effectiveness  and  suitability  of  the  preproduction  NEXRAD  for  DOC,  DOD,  and  DOT  to 
support  a  full-production  decision;  to  review  deficiencies  and  enhancements  documented 
by  the  test  team  during  IOT&E(1A)  and  IOT&E(1B);  to  identify  deficiencies  and 
enhancements  not  previously  documented;  and  to  identify  items  to  be  addressed  during 
follow-on  operational  test  and  evaluation  (FOT&E). 

1.1  AUTHORIZING  DIRECTIVES.  Memorandum  of  Agreement  for  Next  Generation 
Weather  Radar  Initial  Operational  Test  and  Evaluation  Phase  II  (NEXRAD  IOT&E(2)i. 
November  1988;  Next  Generation  Weather  Radar  Test  and  Evaluation  Master  Plan.  March 
1985;  Air  Force  Program  Management  Directive  1058(12)/PE  63707F/64707/351 1 1 F,  March 
1988;  and  Next  Generation  Weather  Radar  (NEXRAD)  Initial  Operational  Test  and 
Evaluation  Phase  II  (IOT&E(2))  Plan  (U).  November  1988. 

1.2  BACKGROUND  OF  OT&E.  In  October  1983,  the  NEXRAD  Program  Council  (NPC) 
requested  that  the  Air  Force  Operational  Test  and  Evaluation  Center  (AFOTEC)  conduct 
the  NEXRAD  IOT&E.  The  NPC  members  and  the  AFOTEC  Commander  signed  a 
memorandum  of  agreement  (MOA)  in  April  1984  outlining  the  specific  responsibilities  of 
AFOTEC  as  the  lead  IOT&E  agency  and  the  associated  IOT&E  responsibilities  of  DOC. 
DOD,  and  DOT.  The  NEXRAD  IOT&E  approach  was  approved  by  the  AFOTEC 
Commander  on  21  August  1984  and  by  the  NPC  on  14  September  1984.  The  NEXRAD 
test  and  evaluation  master  plan  (TEMP)  was  coordinated  and  approved  by  all  participating 
agencies  in  March  1985.  The  TEMP  details  the  responsibilities  of  the  participants  and  the 
general  IOT&E  scenario.  The  NPC  members  and  the  AFOTEC  Commander  signed  a 
second  MOA  on  2  November  1988  which  focused  on  IOT&E(2),  updated  all  agencies 
specific  responsibilities,  and  superseded  the  April  1984  NEXRAD  MOA. 

a.  Between  11  August  and  31  October  1986,  two  independent  test  elements  with 
members  from  DOC,  DOD,  and  DOT,  under  the  overall  management  of  AFOTEC. 
conducted  IOT&E(1A)  of  the  two  competing  contractors’  (Raytheon  and  Unisys,  formerly 
Sperry)  NEXRAD  units.  Each  independent  test  element  identified  a  number  of  deficiencies 
and  enhancements  during  test.  As  a  result  of  these  IOT&E(1  A)  findings  and  other  pertinent 
information,  the  NPC  directed  both  contractors  to  continue  development  and  prepare  for 
additional  testing-IOT&E(1  B). 

b.  Two  independent  test  elements,  again  under  the  overall  management  of  AFOTEC. 
conducted  IOT&E(1B)  of  the  two  competing  contractors’  NEXRAD  units  from  13  April  to 
22  May  1987.  This  test  provided  information  to  the  NPC  as  an  aid  in  selecting  a  single 
contractor  for  the  limited  production  phase  and  identified  a  number  of  deficiencies  that 
required  correction  before  the  start  of  IOT&E(2).  Unisys  was  selected  as  the  limited- 
production  contractor,  and  preparations  for  IOT&E(2)  began. 
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1.3  DESCRIPTION  OF  SYSTEM  TESTED.  During  IOT&E(2),  a  single  validation  phase 
preproduction  NEXRAD  system,  consisting  of  one  Radar  Data  Acquisition  (RDA)  unit,  one 
Radar  Product  Generation  (RPG)  unit,  and  four  Principal  User  Processing  (PUP)  units, 
was  tested.  The  RDA  unit  included  a  Doppler  radar  and  the  software  to  perform  system 
signal  processing,  clutter  suppression,  control,  error  detection,  and  calibration.  The  RPG 
unit  included  the  hardware  and  software  for  real-time  generation,  storage,  and  distribution 
of  radar  products  for  operational  use  and  for  overall  NEXRAD  system  control,  status 
monitoring,  error  detection,  and  data  archiving.  The  PUP  included  the  hardware  and 
software  for  the  request,  display,  storage,  and  annotation  of  products.  It  also  included 
the  hardware  and  software  for  the  control  of  the  PUP,  status  monitoring,  and  data 
archiving.  The  operational,  full-production  NEXRAD  system  is  expected  to  provide  the 
same  capabilities,  but  with  revised  software,  a  higher  data  bit  rate  capability  between  the 
RPG  and  RPG  Operational  Position,  a  production  model  transmitter,  the  hydrology 
functionality,  and  revised  algorithms.  DOC,  DOD,  and  DOT  have  the  option  to  acquire 
approximately  175  radar  systems  and  356  PUPs.  Approximately  160  of  these  systems 
are  planned  to  be  configured  into  a  national  weather  radar  network,  which  would  provide 
radar  coverage  for  the  48  contiguous  states.  Each  NEXRAD  system,  with  the  associated 
communications,  data  processing  hardware  and  software,  display,  and  data  entry 
equipment,  was  designed  to  acquire,  process,  and  distribute  radar  information  on  the 
location,  structure,  intensity,  and  movement  of  weather  phenomena.  The  agencies 
developed  the  operational  concept  of  a  Unit  Radar  Committee  (URC)  to  coordinate  the 
radar  operational  configuration  to  best  support  all  associated  users.  The  test  team 
implemented  this  concept  for  IOT&E(2). 

1.4  TEST  FORCE,  LOCATION,  DATES.  A  160-member,  integrated  tridepartmental  test 
team,  comprised  of  DOC,  DOD,  and  DOT  personnel  under  the  overall  management  of 
AFOTEC,  conducted  an  IOT&E(2)  on  the  Unisys  preproduction  NEXRAD  system.  Agency 
operations,  maintenance,  and  training  specialists  also  contributed  their  expertise  in  test 
activities.  !OT&E(2)  was  divided  into  two  parts  (A  and  B).  Part  A,  combining  development 
test  and  evaluation  (DT&E),  OT&E,  and  contractor  activities,  began  on  6  March  1989  and 
continued  through  7  May  1989.  Part  B  (dedicated  OT&E)  began  on  8  May  1989  and 
continued  through  6  August  1989.  The  test  was  conducted  by  test  team  operators  in  the 
Oklahoma  City  (OKC)  Weather  Service  Forecast  Office  (WSFO)  in  Norman.  Oklahoma 
(DOC  personnel);  the  Base  Weather  Station  (BWS)  at  Tinker  Air  Force  Base  iTAFB). 
Oklahoma  (DOD  personnel);  and  the  Federal  Aviation  Administration  (FAA)  Academy  in 
Oklahoma  City,  Oklahoma  (DOT  personnel).  Maintenance  and  software  personnel 
conducted  test  activities  from  integrated,  tridepartmental  work  centers  located  in  Norman. 
Oklahoma,  with  trips,  as  necessary,  to  the  other  test  sites. 

1.5  CLASSIFICATION  STATEMENT.  There  is  no  classified  information  associated  with 
the  NEXRAD  program. 
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SECTION  II  -  OT&E  DESCRIPTION 


2.0  CRITICAL  OPERATIONAL  ISSUES/OBJECTIVES. 

2.0.1  Critical  Operational  Issues  (COIs).  Five  COIs  were  defined  in  the  TEMP  and 
IOT&E(2)  test  plan  and  are  shown  below.  These  issues  were  reviewed  and  approved  by 
all  participating  agencies  and  by  the  NPC. 

a.  Performance.  Does  NEXRAD  provide  adequate  information  in  a  format  that  will 
allow  DOC,  DOD,  and  DOT  personnel  to  generate  accurate  and  timely  warnings  of 
hazardous  weather  events? 

b.  Availability.  Is  NEXRAD  sufficiently  reliable,  maintainable,  and  logistically 
supportable  to  achieve  the  required  operational  availability? 

c.  Responsiveness.  Does  the  NEXRAD  system  effectively  react  to  multiple  users' 
needs?  Is  the  system  capable  of  processing  many  different  types  of  requests  from  many 
different  users?  Is  the  system  capable  of  processing  high-priority  requests? 

d.  Growth  Capability.  Are  both  hardware  and  software  capable  of  accommodating 
system  expansion  and  update  in  the  future? 

e.  InteroperaDility.  Can  NEXRAD  operate  in  conjunction  with  existing  and  planned 
weather  information  systems/networks? 

2.0.2  Objectives.  The  test  was  based  on  eight  effectiveness,  seven  suitability,  and  three 
combined  effectiveness  and  suitability  objectives  derived  from  the  five  COIs.  Table  11-1, 
paragraph  2.0.3,  is  a  matrix  of  the  COIs  and  objectives.  Definitions  of  the  terms  evaluate, 
assess,  met  requirements,  and  did  not  meet  requirements  are  contained  in  the  glossary 
in  appendix  E.  For  evaluate-level  objectives,  the  test  team  compared  test  data  against 
user-stated  criteria;  for  assess-level  objectives,  the  test  team  collected  and  reported 
information  on  high-interest  areas  without  criteria. 

a.  Objective  E-1.  Evaluate  NEXRAD  as  an  effective  aid  in  preparing  accurate  and 
timely  weather  warnings. 

b.  Objective  E-2.  Evaluate  NEXRAD's  impact  on  operator  workload. 

c.  Objective  E-3.  Assess  whether  current  position  qualifications  for  agency  personnel 
are  adequate  to  effectively  use  NEXRAD. 

d.  Objective  E-4.  Evaluate  NEXRAD  capability  to  provide  required  operational 
support  to  multiple  users. 

e.  Objective  E-5.  Evaluate  NEXRAD  as  an  effective  aid  in  preparing  accurate  and 
timely  weather  advisories. 

f.  Objective  E-6.  Evaluate  NEXRAD  as  an  effective  aid  in  providing  routine  weather 
services. 

g.  Objective  E-7.  Assess  NEXRAD  as  an  effective  aid  to  meeting  agency  mission 
requirements  when  changing  to,  operating  on,  and  recovering  from  backup  power. 

h.  Objective  E-8.  Assess  NEXRAD  electromagnetic  compatibility  (EMC). 
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i.  Objective  ES-9.  Assess  the  adequacy  of  the  planned  NEXRAD  training  to  provide 
the  skills  required  to  effectively  use  and  maintain  NEXRAD. 

j.  Objective  ES-10.  Assess  impacts  of  any  safety  hazards  associated  with  NEXRAD. 

k.  Objective  ES-11.  Assess  factors  impacting  the  interoperability  of  NEXRAD  with 
existing  and  planned  systems. 

l.  Objective  S-12.  Assess  NEXRAD  reliability. 

m.  Objective  S-13.  Evaluate  NEXRAD  maintainability. 

n.  Objective  S-14.  Evaluate  NEXRAD  availability. 

o.  Objective  S-15.  Assess  the  adequacy  of  logistics  support. 

p.  Objective  S-16.  Evaluate  NEXRAD  software  maintainability. 

q.  Objective  S-17.  Assess  the  adequacy  of  planned  and  existing  NEXRAD  software 
support  resources  (SSR). 

r.  Objective  S-18.  Assess  NEXRAD  software  usability. 

2.0.3  COIs.  Table  11-1  contains  a  matrix  of  the  COIs  and  objectives. 

Table  11-1 


Critical  Operational  Issues/Test  Objectives  Matrix 

COIs 

Test  Objectives 

1 

2 

3  4 

5 

E-1 

Weather  Warnings 

X 

X 

E-2 

Operator  Workload 

X 

X 

E-3 

Position  Qualifications 

X 

E-4 

Support  to  Multiple  Users 

X 

E-5 

Weather  Advisories 

X 

X 

E-6 

Routine  Weather  Services 

X 

X 

E-7 

Backup  Power  Operations 

X 

X 

E-8 

Electromagnetic  Compatibility 

X 

X 

ES-9 

Training 

X 

X 

ES-10 

Safety 

X 

X 

X  X 

ES-11 

Interoperability 

X 

S-12 

Reliability 

X 

X 

S-13 

Hardware  Maintainability 

X 

X 

S-14 

Availability 

X 

X 

X 

S-15 

Logistics  Support 

X 

S-16 

Software  Maintainability 

X 

X 

X 

S-17 

Software  Support  Resources 

X 

X 

X  X 

X 

S-18 

Software  Usability 

X 

X 

X 

COI  1  = 

Performance 

COI  2  = 

Availability 

COI  3  = 

Responsiveness 

COI  4  = 

Growth  Capability 

COI  5  = 

Interoperability 
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2.1  SCOPE  AND  METHOD  OF  ACCOMPLISHMENT. 

2.1.1  Scope.  The  duration  of  IOT&E(2)  was  5  months  and  was  divided  into  two  parts 
(A  and  B).  Part  A  was  a  combined  DT&E  and  OT&E  period  which  began  on  6  March 
1989  and  continued  through  7  May  1989.  During  Part  A,  the  use  of  the  NEXRAD  was 
shared  among  DT&E,  OT&E,  and  contractor  activities.  Part  B  began  on  8  May  1989  and 
continued  to  the  end  of  test  on  6  August  1989.  During  the  first  10  weeks  of  Part  B,  the 
NEXRAD  system  was  dedicated  to  operational  testing  24  hours  per  day,  7  days  per  week. 
This  10-week  period  was  the  data  collection  period  for  the  reliability,  maintainability,  and 
availability  (RM&A)  data  calculations.  During  the  last  2  weeks  of  Part  B,  the  test  team 
focused  on  service  report  activities  and  verification  of  PTM  maintenance  procedures  and 
technical  data. 

a.  Operations.  The  test  team  operated  NEXRAD  in  accordance  with  current 
operational  concepts  and  procedures  to  support  actual  operational  missions  of  the  DOC 
and  DOD  and  a  simulated  mission  environment  of  DOT.  For  DOC  and  DOD,  the  NEXRAD 
system  was  used  as  an  aid  in  conducting  their  normal  meteorological  operations  of 
providing  weather  warnings,  weather  advisories,  and  routine  weather  services  support  to 
a  wide  range  of  customers.  The  DOC's  weather  support  area  encompassed  51  counties 
in  Oklahoma,  while  the  DOD’s  responsibility  was  primarily  limited  to  an  area  within  5 
nautical  miles  (nm)  of  Tinker  AFB.  For  the  DOT,  the  NEXRAD  was  used  as  an  aid  in 
conducting  simulated  Center  Weather  Service  Unit  (CWSU)  operations  over  a  geographical 
area  which  encompassed  most  of  Oklahoma.  Test  team  specialists  used  a  separate  PUP 
located  in  the  NEXRAD  Operational  Support  Facility  (OSF).  To  evaluate  NEXRAD's 
capability  to  support  multiple  users,  the  test  team  used  the  three  operational  PUPs,  the 
OSF  PUP,  and  a  unit  loader  to  simulate  an  operationally  representative  processing  load 
of  a  19-user  NEXRAD  site. 

b.  Maintenance.  The  test  team  performed  organizational-level  maintenance  on  the 
system  for  the  entire  5  months  of  the  test.  The  primary  maintenance  activities  were 
troubleshooting  system  malfunctions  to  the  line-replaceable  unit  (LRU)  or  hardware 
components  and  performing  remove  and  replace  actions  or  hardware/software  resets  as 
required.  When  the  test  team  required  assistance  to  complete  the  organizational-level 
maintenance,  the  contractor  was  requested  to  provide  field  maintenance  services  in 
accordance  with  the  Contractor  Support  Services  Plan  (CSSP).  In  addition,  the  contractor 
provided  depot-level  maintenance  support.  Preventive  maintenance  inspections  (PM Is)  were 
conducted  and  evaluated  by  the  test  team. 

c.  Software.  The  test  team  conducted  a  wide  range  of  software  activities.  Selected 
software  documentation  and  source  listings  were  reviewed  to  evaluate  the  maintainability 
of  the  NEXRAD  software.  The  adequacy  of  planned  and  existing  NEXRAD  software 
support  resources  was  also  assessed.  System  documentation  and  implementation 
standards  were  reviewed  for  adequacy  to  support  interoperability  with  existing  and  planned 
systems  (such  as  Automation  of  Field  Operations  and  Services  System.  Automated  Weather 
Distribution  System,  and  Advanced  Weather  Interactive  Processing  System).  The  usability 
of  the  operator  and  maintainer  NEXRAD  software  interfaces  was  also  assessed. 

2.1.2  Questionnaires. 

2.1. 2.1  Operations.  Nine  measures  of  effectiveness  (MOEs)  required  operator  evaluation. 
To  evaluate  these  MOEs,  the  test  team  developed  and  used  several  questionnaires. 
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a.  Operator  Questionnaire.  The  test  team  used  a  1 47-question  Operator 
Questionnaire  to  record  the  opinion  of  operators  on  the  effectiveness  of  NEXRAD  during 
ICT&E(2)  Part  B.  The  Operator  Questionnaire  contained  a  single  general  question  for  each 
primary  MOE.  In  addition,  the  questionnaire  contained  supplemental  questions  that  allowed 
each  operator  to  amplify  conclusions  or  to  point  out  specific  areas  of  concern.  This 
questionnaire  used  a  6-point  response  scale  (see  table  11-2)  for  objectives  E-1,  E-4,  E-5. 
and  E-6  and  a  5-point  scale  (see  table  11-3)  for  objective  E-2.  Operators  were  required 
to  provide  comments  for  the  objectives  that  the  test  team  addressed  at  the  assess  level 
(i.e.,  objectives  E-3,  E-7,  E-8,  and  ES-9). 

Table  11-2 


'Criterion 


Response 

6 

5 

*4 


3 

2 

1 


Operational  Effectiveness  Response  Scale 

Description 
Completely  Effective 
Highly  Effective 
Mildly  Effective 

•  Meets  operator's  minimum  operational 
needs 

Mildly  Ineffective 
Highly  Ineffective 
Completely  Ineffective 
Table  11-3 


Operational  Workload  Response  Scale 


Response 

5 


4 


3 


*2 


Description 

Significant  decrease  in  workload 
•  May  require  less  manning  to  meet 
existing  agency  requirements 

Slight  decrease  in  workload  when  using 
NEXRAD  to  meet  existing  agency 

requirements 

No  change  in  workload  when  using 
NEXRAD  to  meet  existing  agency 

requirements 

Slight  increase  in  workload  when  using 
NEXRAD  to  meet  existing  agency 

requirements 


Criterion 


1 
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Significant  increase  in  workload 
•  May  require  additional  manning  to  meet 
existing  agency  requirements 


b.  Weather  Warning  Questionnaire.  To  evaluate  and  document  the  performance  of 
NEXRAD  for  specific  weather  warning  events,  the  test  team  had  operators  complete 
Weather  Warning  Questionnaires.  The  test  team  reviewed  these  Weather  Warning 
Questionnaires  for  comments  and  reported  specific  aspects  of  NEXRAD  warning  support 
performance. 

c.  School  of  Aerospace  Medicine  (SAM)  Form  202,  Crew  Status  Survey.  To  evaluate 
workload  impacts  during  a  specific  shift,  operators  indicated  on  this  survey  their  peak  and 
average  workload,  subjective  fatigue  at  the  beginning  and  end  of  a  shift,  type  of  weather 
that  occurred  during  the  shift,  and  the  amount  of  unscheduled  activities. 

d.  Operator  Demographics  Questionnaire.  This  questionnaire  provided  the  test  team 
with  each  operator’s  current  job,  qualification  (e.g.,  meteorologist,  weather  officer,  weather 
forecaster,  and  weather  observer),  education  level,  and  years  of  experience. 

e.  Responsiveness  Questionnaire.  Operators  completed  the  Responsiveness 
Questionnaire  to  document  the  demonstrated  responsiveness  of  NEXRAD  during  a  specific 
operator  shift  and  unit  loader  scenario. 

f.  Operations  Training  Questionnaire.  The  test  team  operators  completed  the 
Operations  Training  Questionnaire  to  document  strengths  and  weaknesses  of  the  six- 
phase,  IOT&E(2),  NEXRAD  operations  training  course. 

2. 1.2.2  Maintenance: 

a.  Maintenance  Incident  Questionnaire.  The  test  team  used  the  Maintenance  Incident 
Questionnaire  to  provide  qualitative  information  to  support  the  operational  suitability 
evaluation. 

b.  Maintenance  Training  Questionnaire.  Maintenance  technicians  completed  the 
Maintenance  Training  Questionnaire  to  document  strengths  and  weaknesses  of  provided 
NEXRAD  maintenance  training. 

c.  Training/Skill  Level  Assessment  Questionnaire.  Maintenance  technicians  completed 
the  Training/Skill  Level  Assessment  Questionnaire  to  provide  qualitative  information  on 
overall  training  and  skill  level  requirements. 

2.1 .2.3  Software.  The  test  team  used  six  questionnaires  in  the  evaluation  and  assessment 
of  the  software-related  objectives.  The  questionnaires  used  a  6-point  response  scale  ^ee 
table  11-4). 

a.  Software  Documentation  Questionnaire.  The  test  team  software  evaluators  used 
the  standardized  Software  Documentation  Questionnaire  from  AFOTEC  Pamphlet 
(AFOTECP)  800-2,  volume  III,  Software  Maintainability  Evaluation  Guide,  to  evaluate  the 
NEXRAD  software  documentation. 

b.  Module  Source  Listing  Questionnaire.  The  test  team  software  personnel  used  the 
standardized  Module  Source  Listing  Questionnaire  from  AFOTECP  800-2,  volume  III  to 
evaluate  the  software  source  listing  maintainability. 

c.  Software  Life  Cycle  Process  Questionnaire.  The  test  team  used  a  modified 
Software  Life  Cycle  Processor  Questionnaire  based  on  the  questionnaire  from  AFOTECP 
800-2.  volume  II,  Life  Cycle  Management  Process  Evaluation  Guide,  to  assess  the 
adequacy  of  planned  and  existing  software  support  resources.  The  questionnaire  was 
modified  by  deleting  questions  with  a  focus  on  early  contractor  actions. 
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Table  11-4 


Software  Questionnaire  Response  Scale 


Response 

6 


5 


4 


3 


2 


1 


Description 

COMPLETELY  AGREE:  There  must  be 
absolutely  no  doubt  when  using  this 

response  that  the  characteristic  being 

evaluated  is  totally  satisfactory  with  respect 
to  the  characteristic  addressed. 

STRONGLY  AGREE:  This  response 
indicates  that  the  characteristic  being 

evaluated  is  very  good  and  is  very  helpful 
for  software  supportability. 

GENERALLY  AGREE:  This  response 
indicates  that  the  characteristic  being 

evaluated  is  satisfactory,  but  may  require 
improvements  to  make  it  helpful  for  software 
supportability. 

GENERALLY  DISAGREE:  This  response 
indicates  that  the  characteristic  being 
evaluated  is  unsatisfactory,  and  some 
improvements  are  required  to  make  it  helpful 
for  software  supportability. 

STRONGLY  DISAGREE:  This  response 
indicates  that  the  characteristic  being 
evaluated  is  unsatisfactory,  and  major 
improvements  are  required  before  it  would 
be  helpful  for  software  supportability. 

COMPLETELY  DISAGREE:  There  must 
be  absolutely  no  doubt  when  using  this 
response  that  the  characteristic  being 
evaluated  is  totally  unsatisfactory  with 
respect  to  the  characteristic  addressed. 


Averages  of  3.5  and  above  indicate  generally  favorable  characteristics. 


d.  Software  Support  Resources  (SSR)  Evaluation  Questionnaire.  The  test  team  used 
the  standardized  SSR  Evaluation  Questionnaire  from  AFOTECP  800-2.  volume  V.  Software 
Support  Resources  Questionnaire,  to  assess  the  adequacy  of  planned  and  existing  software 
support  resources. 

e.  Software  Usability  Questionnaire  (SUQ).  The  test  team  used  the  standardized 
SUQ  from  AFOTECP  800-2,  volume  IV,  Software  Usability  Evaluation  Guide,  to  assess 
the  usability  of  the  system  software  interfaces. 
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f.  Software  Training  Questionnaire.  The  test  team  used  a  Software  Training 
Questionnaire  to  assess  the  provided  7-week  software  training  course  and  the  planned 
software  training. 

2.1.3  Questionnaire  Administration  and  Application. 

2. 1.3.1  Operations.  The  operational  effectiveness  objectives  relied  on  the  opinion  of  the 
NEXRAD  operators  obtained  through  questionnaires. 

a.  Operator  Questionnaire.  The  Operator  Questionnaire  was  the  primary  evaluation 
tool  used  by  the  test  team  to  evaluate  NEXRAD  against  the  effectiveness  objectives. 
Additional  data  were  collected  and  statistics  calculated  for  supporting  MOEs.  These 
additional  data  are  not  discussed  in  this  report  except  in  those  cases  where  the  results 
conflicted  with  the  median  responses  determined  from  the  Operator  Questionnaire.  When 
administering  the  Operator  Questionnaire,  the  test  team  used  a  structured  interview  process 
to  focus  the  operator's  responses  on  the  test  objectives  and  to  encourage  discussion  after 
each  question  had  been  rated.  Written  comments  on  each  question  were  strongly 
encouraged  to  provide  additional  insight  into  the  operator’s  evaluation.  During  the  Operator 
Questionnaire  administration  at  the  end  of  Part  B,  operators  were  instructed  to  answer  each 
question  twice.  They  were  instructed  to  base  their  first  response  on  their  experience  when 
the  system  was  operating  regardless  of  system  outages.  For  their  second  response,  the 
evaluators  were  instructed  to  consider  overall  system  performance  including  the  impact  of 
system  outages  during  IOT&E(2)  Part  B.  For  clarity  and  to  separate  effectiveness  and 
suitability  issues,  the  questionnaire  responses  based  on  when  the  system  was  operating 
were  used  as  the  sole  measure  in  evaluating  NEXRAD  against  users’  requirements. 
Therefore,  the  results  and  conclusions  for  the  effectiveness  objectives  reflect  the  operators' 
opinions  of  NEXRAD’s  effectiveness  only  when  the  system  was  operating.  The  operators’ 
responses  which  addressed  overall  system  performance  (including  impacts  of  availability) 
will  be  discussed  in  paragraph  3.19. 

b.  Weather  Warning  Questionnaire.  Following  any  operations  shift  during  which 
severe  or  potentially  severe  weather  occurred  in  or  near  their  area  of  warning  responsibility, 
operators  completed  a  Weather  Warning  Questionnaire  for  the  event.  These  Weather 
Warning  Questionnaire  responses  for  specific  events  were  used  to  supplement  the  pnmary 
weather  warning  MOE  data  obtained  through  the  Operator  Questionnaire. 

c.  SAM  Form  202,  Crew  Status  Survey.  The  SAM  Form  202  was  administered  to 
each  operator  at  the  end  of  each  shift  during  Part  B.  Data  from  incomplete  forms  or  when 
the  radar  was  down  for  maintenance  were  excluded  from  analysis.  An  analysis  of  these 
data  was  used  to  supplement  the  primary  workload  MOE  data  obtained  through  the 
Operator  Questionnaire. 

d.  Operator  Demographics  Questionnaire.  Before  testing  began,  the  test  team 
collected  operator  qualification  information  by  administering  the  Operator  Demographics 
Questionnaire  to  each  operator. 

e.  Responsiveness  Questionnaire.  Operators  completed  this  questionnaire  at  the  end 
of  each  shift  during  Part  B. 

f.  Operations  Training  Questionnaire.  Operators  completed  the  training  questionnaire 
at  the  end  of  phase  2  of  the  IOT&E(2)  NEXRAD  operations  training  course,  at  the  end 
of  phase  4,  at  the  end  of  phase  6  (DOD  only),  and  prior  to  the  end  of  the  test  (DOD  only). 
Agency  training  specialists  reviewed  completed  questionnaires  to  support  the  assessment 
of  the  adequacy  of  planned  agency  NEXRAD  operations  training. 


11-7 


2. 1.3.2  Maintenance: 


a.  Maintenance  Incident  Questionnaire.  The  maintenance  technicians  completed  a 
Maintenance  Incident  Questionnaire  after  each  maintenance  action.  Responses  from  the 
questionnaire  were  examined  to  aid  in  the  assessment  of  safety,  diagnostics,  training, 
support  equipment,  spares,  and  any  other  area  affecting  maintainability. 

b.  Maintenance  Training  Questionnaire.  The  maintenance  technicians  completed  the 
Maintenance  Training  Questionnaire  after  each  block  of  instruction  during  the  7-week 
course.  The  test  team  used  the  questionnaire  results  in  the  qualitative  assessment  of 
maintenance  training. 

c.  Training/Skill  Level  Assessment  Questionnaire.  The  maintenance  technicians 
completed  the  Training/Skill  Level  Assessment  Questionnaire  at  the  end  of  training  and 
again  prior  to  the  end  of  their  participation  in  IOT&E(2).  Responses  were  used  as  part 
of  the  overall  assessment  of  the  training  and  skill  level  requirement. 

2.1 .3.3  Software.  The  evaluation  and  assessment  of  the  software-related  objectives  relied 
on  the  expert  opinion  of  the  test  team  evaluators  obtained  through  questionnaire  responses 
and  written  comments.  IOT&E(2)  results  were  based  on  the  average  evaluator  responses 
on  the  questionnaires. 

a.  Software  Documentation  Questionnaire.  The  documentation  of  four  computer 
program  configuration  items  (CPCIs)  was  evaluated  by  10  software  evaluators  using  the 
Software  Documentation  Questionnaire.  The  average  of  the  questionnaire  responses  and 
the  written  comments  were  used  to  form  the  basis  of  the  software  documentation 
evaluation. 

b.  Module  Source  Listing  Questionnaire.  One  hundred  seventy-nine  randomly 
selected  modules  were  evaluated  by  two  teams  of  five  software  evaluators  each,  using 
the  Module  Source  Listing  Questionnaire.  The  average  of  the  questionnaire  responses 
and  the  written  comments  were  used  to  form  the  basis  of  the  software  source  listing 
evaluation. 

c.  Software  Life  Cycle  Process  Questionnaire.  Using  the  information  available  at 
the  time  of  the  test.  10  software  evaluators  completed  the  Software  Life  Cycle  Process 
Questionnaire.  Discussion  was  encouraged  to  ensure  and  focus  understanding  of  each 
question.  The  questionnaire  responses  were  used  to  identify  trends  that  were  then 
reinforced  by  the  written  comments  to  form  the  basis  of  the  software  life  cycle  process 
assessment. 

d.  Software  Support  Resources  Evaluation  Questionnaire.  Using  the  information 
available  at  the  time  of  the  test,  10  software  evaluators  completed  the  Software  Support 
Resources  Evaluation  Questionnaire.  Discussion  was  encouraged  to  ensure  and  focus 
understanding  of  each  question.  The  questionnaire  responses  were  used  to  identify  trends 
that  were  then  amplified  by  the  written  comments  to  form  the  basis  of  the  software  support 
resources  assessment. 

e.  Software  Usability  Questionnaire.  The  SUQ  was  administered  to  the  operators 
and  maintainers  using  a  structured  interview  process  to  focus  the  responses  on  the 
question  objective.  Discussion  was  encouraged.  The  questionnaire  responses  and  the 
written  comments  were  used  to  form  the  basis  of  the  software  usability  assessment. 


f.  Software  Training  Questionnaire.  The  Software  Training  Questionnaire  was 
administered  after  each  block  of  instruction  during  the  7-week  software  training  course. 
The  questionnaire  responses  were  used  to  identify  trends  that  were  then  amplified  by  the 
written  comments  to  form  the  basis  of  the  software  training  assessment. 

2.1.4  Supporting  Data  Document  (SDD).  The  SDD  is  a  separate  document  that  provides 
an  index  to  the  summary  statistics  {e.g..  operator  questionnaire  response  histograms)  and 
raw  data  collected,  compiled,  analyzed,  and  written  for  all  primary  and  supporting  MOEs. 
It  was  prepared  by  the  test  team  and  is  maintained  by  HQ  AFOTEC/RS.  Requests  for 
these  basic  data  should  be  directed  to  that  office. 

2.2  PLANNING  CONSIDERATIONS  AND  LIMITING  FACTORS. 

2.2.1  Planning  Considerations.  Planning  considerations  which  affected  the  scope  or 
conduct  of  IOT&E(2)  are  listed  below: 

a.  Acquisition  Baseline.  A  formal  acquisition  baseline  and  corresponding 
documentation  were  not  required  contractor-deliverable  items  for  IOT&E(2).  The  test  team 
conducted  IOT&E(2)  with  a  baseline  determined  by  the  minimum  test  start  criteria  and  the 
status  or  the  system  at  test  start. 

b.  Simulated  NEXRAD  Network.  The  test  team  used  only  four  PUPs  and  a  single 
radar  unit.  Network  demands  were  simulated  through  dial-up  communications  lines  and 
a  unit  loader. 

c.  Use  of  Existing  Weather  Radar.  There  was  a  legal  requirement  to  use 
commissioned  radars  to  meet  information  dissemination  requirements.  Therefore,  DOC 
and  DOD  continued  to  refer  to  existing  weather  radars  to  provide  weather  services.  The 
IOT&E(2)  of  the  NEXRAD  unit  was  an  added  requirement  to  existing  duties  with  no 
reduction  in  the  requirement  to  use  the  existing  radars. 

d.  Limited  Maintenance  On-The-Job  Training  (OJT).  The  agencies'  training  concept 
for  maintenance  specified  that  trained,  expenenced  personnel  would  provide  OJT  to  all 
maintenance  technicians  following  contractor-provided  training.  Since  this  was  a  new 
system,  a  body  of  trained,  experienced  government  personnel  was  not  available  to  fulfill 
this  function. 

2.2.2  Limiting  Factors.  Limitations  to  the  test  team's  ability  to  conduct  a  completely 
realistic  IOT&E(2)  are  listed  below.  Despite  these  limitations,  the  test  emphasized 
operational  realism  throughout  IOT&E(2)  to  the  maximum  extent  possible.  Each  limitation 
should  be  addressed  in  the  FOT&E  outlined  in  the  NEXRAD  TEMP  (March  1985). 

a.  Validation  Phase  Preproduction  System.  The  system  tested  did  not  include  certain 
capabilities  (e.g.,  hydrology  functionality,  higher  data  rate  capability  between  the  RPG  and 
RPG  Operational  Position,  revised  production  phase  algorithms,  and  production  model 
transmitter)  to  be  implemented  in  the  limited  or  full-scale  production  phases.  Therefore, 
full  system  capability  could  not  be  determined. 

b.  Limited  Integrated  Logistics  Support.  The  provisioning  process  for  spares  and 
support  equipment  was  not  completed  before  the  end  of  the  test.  The  test  team  used  a 
limited  contractor-proposed,  Joint  System  Program  Office  (JSPO)-approved  spares  and 
support  equipment  package  and  a  PTM.  Therefore,  the  system's  future  operational  mean 
downtime  and  the  adequacy  of  the  spares  and  support  equipment  concepts  could  not  be 
determined. 
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c.  Limited  DOT  Operational  Work  Environment.  Most  of  the  support  equipment  that 
is  normally  part  of  an  Air  Route  Traffic  Control  Center,  CWRU,  and  Flow  Control  Unit  was 
not  available  at  the  simulated  CWSU  facility  at  the  FAA  +<.i  emy.  In  addition,  most  real¬ 
time  mission  responsibilities  could  not  be  simulated,  i  nerefore,  the  effectiveness  of 
NEXRAD  as  an  aid  to  an  operational  CWSU  could  not  be  determined. 

d.  Software  Support  Resources.  Government  software  support  resources  were  not 
in  place.  Software  support  plans  and  procedures,  scheduled  to  be  published  after 
IOT&E(2),  were  not  available.  Therefore,  the  government  procedures  for  software  support 
could  not  be  fully  evaluated. 

2.3  CONTRACTOR  INVOLVEMENT: 

a.  Unisys  (the  primary  contractor)  and  two  of  their  subcontractors  (Westinghouse  and 
Concurrent  Computer  Corporation)  provided  field  maintenance  services  (on  an  "as  required" 
basis  to  complete  organizational-level  maintenance)  and  depot-level  maintenance  support 
in  accordance  with  the  approved  CSSP.  This  was  in  accordance  with  the  maintenance 
concept.  Unisys  also  provided  operations,  maintenance,  and  software  training  for  IOT&E 
personnel.  Concurrent  Computer  Corporation  and  Westinghouse  provided  additional 
maintenance  training  to  test  team  technicians. 

b.  Maintenance  actions  performed  by  contractors  were  observed  over-the  shoulder 
by  test  team  maintenance  technicians  and  were  documented  on  maintenance  data 
collection  (MDC)  forms.  This  ensured  that  contractor  involvement  was  within  the  framework 
of  the  maintenance  concept.  All  data  collection  and  processing  were  done  by  the  test  team 
to  ensure  data  and  analysis  integrity.  All  identified  system  deficiencies  have  been  reported 
through  the  service  report  (SR)  process  in  accordance  with  Technical  Order  (TO)  00-35D- 
54,  USAF  Materiel  Deficiency  Reporting  and  Investigating  System. 


SECTION  III  -  OPERATIONAL  EFFECTIVENESS  AND  SUITABILITY 


3.0  SUMMARY.  NEXRAD  was  an  effective  aid  in  providing  weather  warning  (objective 
E-1),  weather  advisory  (objective  E-5),  and  routine  weather  services  (objective  E-6)  support. 
Although  there  was  only  a  slight  increase  in  operator  workload  when  operators  used  the 
PUP  only,  there  was  a  significant  increase  in  operator  workload  when  operators  used  the 
PUP  and  Unit  Control  Position  (UCP)  together  (objective  E-2).  Operators,  specialists,  and 
supervisors  stated  that  existing  agency  position  qualifications  were  adequate  for  using 
NEXRAD  (objective  E-3).  The  system  was  responsive  in  providing  required  products  when 
operating  under  a  representative  maximum  user  load  (objective  E-4).  NEXRAD  often  failed 
to  recover  automatically  from  backup  power  transitions  (objective  E-7).  The  test  team 
noted  one  apparent  EMC  problem--a  wavy  presentation  on  the  RDA  application  terminal 
throughout  IOT&E(2)  (objective  E-8).  Numerous  deficiencies  were  identified  in  planned 
agency  operations,  maintenance,  and  software  training  (objective  ES-9).  The  t^st  team 
identified  and  documented  56  safety  deficiencies,  9  of  which  had  the  potential  to  cause 
death,  severe  injury,  or  major  system  damage  (objective  ES-10).  Deficiencies  in  the 
interface  documentation  made  it  difficult  and  time-consuming  for  test  team  members  to  find 
and  organize  interoperability  information  (objective  ES-1 1).  The  system  mean  time  between 
maintenance  (MTBM)  (total  corrective)  was  25.3  hours;  four  reliability  problems  were 
identified  with  the  preproduction  transmitter  RPG,  graphics  processor,  and  optical  disk  drive 
units  (objective  S-12).  Demonstrated  NEXRAD  maintainability,  fault  isolation  (objective 
S-1 3),  and  availability  (objective  S-14)  did  not  meet  the  users'  requirements.  Numerous 
deficiencies  were  identified  with  support  equipment,  sparing,  and  the  PTM  (objective  S-1 5). 
The  evaluated  software  documentation  and  source  code  listings  for  four  CPCIs  met  the 
users'  requirements  (objective  S-1 6);  however,  serious  deficiencies  were  identified  with 
the  overall  documentation.  There  is  a  risk  that  the  existing  and  planned  software  support 
resources  may  not  be  adequate  for  the  government  to  assume  software  support 
responsibility  (objective  S-1 7).  The  test  team  assessed  the  usability  of  six  NEXRAD 
software  interfaces  and  identified  several  deficiencies  (objective  S-1 8). 

3.1  OBJECTIVE  E-1.  Evaluate  NEXRAD  as  an  effective  aid  in  preparing  accurate  and 
timely  weather  warnings. 

3.1.1  Method.  To  evaluate  this  objective.  DOC  and  DOD  operators  used  NEXRAD 
information  to  assist  in  preparing  operational  weather  warnings  during  IOT&E(2).  The  test 
team  used  the  Weather  Warning  Questionnaire  and  the  Operator  Questionnaire  to 
document  individual  warning  events  and  general  NEXRAD  performance,  respectively,  during 
IOT&E(2)  Part  B.  In  addition,  the  test  team  collected  weather  warning  verification  statistics 
and  specialists  reports  to  support  the  operators'  evaluation. 

3.1. 1.1  Weather  Warning  Procedures.  NEXRAD  was  operated  and  evaluated  using 
existing  agency  procedures  and  requirements.  DOC  meteorologists  issued  warnings  for 
51  counties  within  Oklahoma  as  prescribed  in  Weather  Service  Operations  Manual  (WSOM) 
chapter  C-47  and  station  duty  manuals.  DOD  forecasters  issued  warnings  within  a  5  nm 
radius  of  Tinker  AFB  in  accordance  with  OC-ALC-TAFB  Regulation  105-1 .  Weather  Support, 
and  current  standard  operating  procedures  (SOPs).  DOT  meteorologists  do  not  issue 
weather  warnings  as  part  of  their  existing  agency  support. 

3.1 .1 .2  Operator  Questionnaire.  The  test  team  used  the  Operator  Questionnaire  to  record 
the  opinions  of  operators  on  NEXRAD  as  an  effective  aid  for  weather  warnings.  The 
Operator  Questionnaire  was  administered  at  the  end  of  Part  B.  The  rating  for  this  objective 
was  based  on  the  operators'  median  response  to  the  question  "What  was  the  overall 
effectiveness  of  NEXRAD  as  an  aid  for  you  in  preparing  accurate  and  timely  weather 


warnings?"  (question  number  1,  appendix  F).  The  test  team  compared  this  median 
response  to  the  criterion.  The  criterion  was  a  median  response  of  4  or  greater  on  a  6- 
point  scale  (ranging  from  1  =  completely  ineffective  to  6  =  completely  effective).  A  score 
of  4  or  greater  would  indicate  that  N  EX  RAD  was  an  effective  aid  in  preparing  weather 
warnings. 

3.1. 1.3  Weather  Warning  Questionnaire.  To  evaluate  and  document  the  performance  of 
NEXRAD  for  specific  weather  warning  events,  the  test  team  had  operators  complete 
Weather  Warning  Questionnaires.  Following  any  operations  shift  during  which  severe  or 
potentially  severe  weather  occurred  in  or  near  their  area  of  warning  responsibility,  operators 
completed  a  Weather  Warning  Questionnaire  for  that  event.  The  test  team  reviewed  these 
Weather  Warning  Questionnaires  for  comments  and  reported  specific  aspects  of  NEXRAD 
warning  support  performance.  These  Weather  Warning  Questionnaire  responses  for 
specific  events  were  used  to  supplement  the  primary  MOE  data  obtained  through  the 
Operator  Questionnaire. 

3. 1.1. 4  Weather  Warning  Verification.  The  test  team  made  use  of  existing  agency 
verification  networks  and  specialists  to  verify  the  success  or  failure  of  weather  warnings. 
The  test  team  used  DOC's  verification  network  that  consisted  of  over  2,500  trained  severe 
weather  spotters.  Doppler  radar  specialists  from  the  National  Severe  Storms  Laboratory 
(NSSL)  used  the  PUP  located  in  the  OSF  to  compare  NEXRAD  product  performance  with 
observed  weather.  To  gain  additional  weather  warning  verification  data,  one  to  three  storm 
intercept  teams  were  often  sent  out.  In  addition,  following  significant  weather  events  when 
the  severity  of  the  storms  were  uncertain,  the  test  team  sent  out  storm  damage  survey 
teams  to  collect  verification  data. 

3. 1.1. 5  Weather  Warning  Statistics.  Based  on  the  weather  warning  verification  data,  the 
test  team  calculated  weather  warning  statistics  to  support  the  operator’s  evaluation.  These 
weather  warning  statistics  were  probability  of  detection  (POD);  false  alarm  rate  (FAR); 
critical  success  index;  and  capability,  warning,  and  event  lead  times.  The  glossary  in 
appendix  E  provides  a  definition  of  these  statistics. 

3.1.2  Results  and  Conclusions.  NEXRAD  met  the  users' requirements  for  weather  warning 
support.  The  Operator  Questionnaire  median  response  tor  the  22  DOC  and  DOD  operators 
was  4  (mildly  effective,  meets  operator’s  minimum  operational  needs).  During  IOT&E(2). 
the  1 1  DOC  operators  used  NEXRAD  as  an  aid  to  issue  681  weather  warnings  within  their 
51 -county  warning  area  in  Oklahoma.  The  11  DOD  operators  used  NEXRAD  as  an  aid 
to  issue  44  weather  warnings  for  the  area  within  5  nm  of  Tinker  AFB.  Severe  weather 
was  reported  within  at  least  one  of  these  two  warning  areas  on  55  of  the  154  test  days. 

3.1. 2.1  Operators  stated  that  NEXRAD  met  their  minimum  operational  requirements  as 
an  aid  in  preparing  accurate  and  timely  weather  warnings.  This  effectiveness  was  achieved 
primarily  because  of  the  high  resolution  and  accuracy  of  the  NEXRAD  reflectivity-based 
products.  The  high  resolution  allowed  operators  to  determine  the  internal  storm  structure 
and  better  understand  the  atmospheric  conditions.  The  reflectivity-based  products  aided 
the  operators  in  preparing  severe  thunderstorm  warnings  throughout  their  area  of  warning 
responsibility.  The  capability  to  magnify  and  time-lapse  storms  in  high-color  resolution,  the 
use  of  background  maps,  and  the  use  of  the  reflectivity-based  Vertically  Integrated  Liquid 
Water  (VIL)  product  were  particularly  effective.  With  NEXRAD,  DOC  operators  were  able 
to  accurately  specify  the  counties  or  parts  of  counties  included  in  a  warning  area  and  the 
time  duration  for  the  warning  event. 
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3. 1.2.2  however,  the  test  team  identified  deficiencies  that  impacted  the  operational 
effectiveness  of  N  EX  RAD  for  weather  warning  support.  During  widespread  convective 
activity,  velocity-based  products  were  often  severely  degraded  by  large  areas  of  range- 
folded  and  incorrectly  dealiased  data  (see  Glossary,  appendix  E).  DOC  operators  stated 
that  these  severely  degraded  velocity-based  products  were  highly  ineffective  as  an  aid  in 
analyzing  storms  for  their  tornadic  potential.  In  many  cases,  winds  associated  with  gust 
fronts  approaching  Oklahoma  City  and  Tinker  AFB  were  masked  by  range-folded  data 
from  second  trip  echoes.  DOD  plans  specified  that  DOD  radars  will  be  sited  10  to  35 
miles  from  a  DOD  installation  in  the  direction  of  the  prevailing  storm  track  such  that  the 
storms  normally  cross  over  the  installation  before  reaching  the  radar.  Therefore,  unless 
corrected,  range-folded  data  will  likely  mask  most  winds  associated  with  gust  fronts 
approaching  DOD  installations.  Additionally,  the  incorrectly  dealiased  velocity  fields  and 
the  current  state  of  the  mesocyclone  detection  and  hail  algorithms  resulted  in  numerous 
false  severe  weather  indications.  For  example,  Doppler  radar  specialists  from  NSSL 
estimated  the  false  alarm  rate  associated  with  the  mesocyclone  detection  algorithm  was 
greater  than  50  percent.  Further,  the  DOC  operators  could  not  locate  severe  storms  with 
respect  to  Oklahoma  cities  and  towns  with  the  contractor-provided  background  maps.  To 
overcome  this  deficiency,  DOC  operators  deduped  a  city  background  map  of  sufficient 
detail  to  prepare  accurate  weather  warnings. 

3.1.3  Recommendations.  For  JSPO: 

a.  Eliminate  the  impact  of  range-folded  data  on  the  velocity-based  products. 
(SR  219) 

b.  Provide  an  effective  velocity  dealiasing  algorithm.  (SRs  500,  062B,  441 ,  507) 

c.  Provide  reliable  hail  and  mesocyclone  algorithm  outputs.  (SRs  208,  380,  228A) 

d.  Provide  complete  background  maps  with  adequate  detail.  (SRs  050,  349.  238A. 
281,  122.  433) 

3.2  OBJECTIVE  E-2.  Evaluate  NEXRAD's  impact  on  operator  workload. 

3.2.1  Method.  To  evaluate  this  objective,  NEXRAD  was  operated  by  BWS.  WSFO.  and 
CWSU  meteorologists,  forecasters,  and  observers  using  existing  agency  procedures  and 
requirements.  The  test  team  used  the  Operator  Questionnaire  to  evaluate  general  workload 
impacts  during  IOT&E(2)  Part  B.  The  test  team  used  the  SAM  Form  202.  Crew  Status 
Survey,  to  document  individual  workload  impacts  during  specific  operator  shifts.  In  addition, 
site  supervisors  and  agency  specialists  commented  on  their  observations  of  workload 
impacts  during  IOT&E(2). 

3.2.1. 1  Operational  Procedures.  To  evaluate  NEXRAD’s  impacts  on  operator  workload, 
test  team  operators  used  NEXRAD  as  an  aid  to  prepare  weather  warnings,  weather 
advisories,  and  routine  weather  services  using  existing  agency  requirements  and 
procedures.  Specific  operational  procedures  are  provided  in  paragraph  3.1 .1 .1  for  warnings, 
paragraph  3.5.1. 1  for  advisories,  and  paragraph  3. 6. 1.1  for  routine  services.  Since  this 
objective  addressed  NEXRAD’s  impact  on  operator  workload  to  meet  only  existing  agency 
requirements,  workload  associated  with  the  new  radar-coded  message  (RCM)  requirement 
is  addressed  separately  in  objective  E-6  (routine  services). 

3.2. 1.2  Workload  Impacts.  Operators  addressed  the  workload  impact  of  two  NEXRAD 
activities:  obtaining  and  interpreting  meteorological  products  at  the  PUP  and  controlling 
NEXRAD  equipment  using  the  UCP.  The  test  team  evaluated  the  impacts  of  both  of  these 
activities.  Each  test  site  had  a  PUP.  However,  in  order  to  allow  both  the  DOC  and  DOD 
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operators  to  use  the  UCP  during  IOT&E(2),  the  test  team  positioned  the  UCP  at  Tinker 
AFB  for  the  first  6  weeks  of  Part  B  and  at  the  OKC  WSFO  for  the  remainder  of  the  test. 
During  the  first  6  weeks  of  Part  B,  the  DOC  and  DOT  operators  evaluated  the  workload 
impact  of  the  PUP  only,  while  the  DOD  evaluated  both  the  PUP  and  UCP  together.  For 
the  remainder  of  IOT&E(2),  the  DOC  operators  evaluated  the  workload  impact  of  the  PUP 
and  UCP  together,  while  the  DOD  and  DOT  evaluated  the  PUP  only.  Current  agency 
plans  do  not  require  DOT  operators  to  use  a  UCP. 

3.2.1. 3  Operator  Questionnaire.  To  evaluate  the  two  primary  MOEs,  the  test  team 
administered  the  Operator  Questionnaire  to  all  test  team  operators  in  a  structured  interview 
process  (see  paragraphs  2.1. 1.1  and  2. 1.2.1).  Using  a  5-point  scale,  operators  evaluated 
workload  impacts  as  the  result  of  using  the  NEXRAD  PUP  only.  Using  the  same  5-point 
scale,  operators  then  evaluated  workload  impacts  of  using  both  the  NEXRAD  PUP  and 
UCP  together.  The  ratings  for  this  objective  were  based  on  the  operators’  median 
response  to  these  two  questions:  (a)  "What  was  the  impact  on  workload  when  you  used 
the  NEXRAD  PUP  to  perform  existing  agency  requirements?"  (b)  "What  was  the  impact 
on  workload  when  you  used  the  NEXRAD  PUP  and  UCP  to  perform  existing  agency 
requirements?"  (question  numbers  2  and  3,  appendix  F).  The  test  team  compared  the 
median  response  with  the  users’  criterion.  The  criterion  for  each  MOE  was  a  median 
response  of  2  or  greater  on  the  5-point  scale.  A  score  of  2  or  greater  would  indicate  that 
there  was  no  significant  increase  in  workload  when  using  NEXRAD  in  performing  agency 
requirements. 

3.2.1. 4  School  of  Aerospace  Medicine  Form  202.  The  SAM  Form  202,  Crew  Status 
Survey,  was  administered  to  each  operator  at  the  end  of  each  shift  during  Part  B.  On 
this  form,  the  operators  indicated  their  peak  and  average  workload,  subjective  fatigue  at 
the  beginning  and  end  of  a  shift,  type  of  weather  that  occurred  during  the  shift,  and  the 
amount  of  unscheduled  activities.  Data  from  incomplete  forms  or  when  the  radar  was 
down  for  maintenance  were  excluded  from  analysis.  An  analysis  of  these  data  was  used 
to  supplement  the  primary  MOE  data  obtained  through  the  Operator  Questionnaire. 

3.2.1. 5  Site  Supervisors.  Site  supervisors  at  the  OKC  WSFO  and  the  Tinker  AFB  BWS 
(meteorologist  in  charge  (MIC),  deputy  MIC,  detachment  commander,  and  chief  of  weather 
station  operations)  provided  the  test  team  with  end-of-test  reports  during  the  last  2  weeks 
of  Part  B.  Part  of  each  report  addressed  the  workload  impacts  of  NEXRAD  on  forecast 
office/base  weather  station  operations. 

3.2. 1.6  Agency  Specialists.  Specialists  from  the  agency  and  regional  headquarters 
observed  NEXRAD  operations  at  the  operational  test  sites.  At  the  end  of  the  5  to  10  day 
review,  these  specialists  provided  trip  reports  addressing  the  impact  of  NEXRAD  on 
operator  workload.  The  test  team  used  these  reports  to  supplement  the  operators' 
evaluation. 

3.2.2  Results  and  Conclusions.  NEXRAD  met  the  users’  requirement  for  operator 
workload  when  the  NEXRAD  PUP  alone  was  used  to  perform  existing  agency  weather 
support  activities.  The  Operator  Questionnaire  median  response  for  29  operators  from  the 
three  agencies  was  2  (slight  increase  in  workload).  The  operator  workload  when  the  UCP 
and  PUP  were  used  together  did  not  meet  the  users’  requirement.  The  Operator 
Questionnaire  median  response  for  26  DOC  and  DOD  operators  was  1  (significant  increase 
in  workload). 

3.2.2. 1  PUP  Only  Workload  Impacts.  DOC  and  DOD  operators  found  that  using  the 
NEXRAD  PUP  alone  to  meet  existing  agency  requirements  resulted  in  a  slight  increase 
in  operator  workload  while  DOT  operators  found  it  would  produce  a  significant  increase 
in  their  workload. 


a.  DOC  and  DOD  operators  stated  that  to  adequately  examine  the  NEXRAD-provided 
information,  additional  radar  interpretation  time  was  required.  However,  these  operators 
also  stated  that  the  slight  increase  in  workload  was  accompanied  with  an  increased 
understanding  of  the  current  state  of  weather  conditions.  In  addition,  DOC  and  DOD 
operators  experienced  a  slight  increase  in  workload  as  a  result  of  the  initial  unfamiliarity 
with  many  NEXRAD  functions.  DOC  and  DOD  operators  found  that,  as  with  their  existing 
weather  radars,  using  NEXRAD  in  potentially  severe  weather  situations  required  a  dedicated 
operator. 

b.  DOT  operators  stated  that  manually  acquiring  and  examining  products  from  multiple 
NEXRADs  would  produce  a  significant  increase  in  their  workload.  The  Operator 
Questionnaire  median  response  for  three  DOT  operators  was  1.  The  time  required  to  dial 
individual  RPGs  and  perform  one-time  requests  would  likely  degrade  the  effectiveness  of 
CWSU  meteorologists'  ability  to  meet  their  mission  requirements  if  the  planned  Real-time 
Weather  Processor  is  unable  to  automate  this  function. 

3.2.2.2  PUP  and  UCP  Workload  Impacts.  Operators  stated  that  using  the  UCP  and  the 
PUP  together  produced  a  significant  increase  in  operator  workload.  Operators  noted  that 
this  was  primarily  the  result  of  required  system  responsibilities  to  support  the  multiple- 
user  radar  configuration.  These  responsibilities  included  system  status  monitoring, 
maintenance  problem  identification,  maintainor  notification,  environmental  wind  updating, 
pulse  repetition  frequency  changes,  associated  users  coordination,  and  volume  coverage 
pattern  selection.  The  UCP  duties  were  sometimes  delayed  or  not  performed  because 
of  other  mission  requirements.  This  often  resulted  in  the  NEXRAD  system  operating  in 
a  mode  that  was  not  optimal  for  the  existing  weather  conditions.  Because  of  limitations 
associated  with  the  free  text  message  (FTM)  functionality,  operators  at  the  UCP  site  were 
frequently  interrupted  from  mission  duties  to  respond  to  telephone  calls  from  the  other  two 
associated  PUP  sites  or  to  initiate  calls  to  them.  Operators  also  stated  that  the 
inconsistent  UCP  and  PUP  menus  and  problems  with  UCP  keystroke  entry  further  hindered 
the  accomplishment  of  UCP  duties.  Results  from  the  SAM  Form  202  analysis  and  site 
supervisor  reports  also  indicated  that  there  was  an  increase  in  operator  workload  when 
the  UCP  was  at  the  operator’s  test  site. 

3.2.3  Recommendations.  For  JSPO: 

a.  Correct  deficiencies  with  the  UCP  user  interface.  (SRs  530.  168A.  009A.  167. 
402,  173,  082B.  337.  164,  069.  177.  175,  174) 

b.  Correct  deficiencies  associated  with  the  FTM  functionality.  (SRs  027B.  1 78.  446, 
447,  445) 

c.  Correct  deficiencies  associated  with  the  dialup  interface.  (SRs  395,  394,  515) 

3.3  OBJECTIVE  E-3.  Assess  whether  current  position  qualifications  for  agency  personnel 
are  adequate  to  effectively  use  NEXRAD. 

3.3.1  Method.  To  assess  position  qualification  requirements,  the  test  team  collected 
operator  demographics  information  and  comments  from  operators,  site  supervisors,  and 
agency  specialists. 

3.3. 1.1  Operator  Demographics  Questionnaire.  Before  testing  began,  the  test  team 
collected  operator  qualification  information  by  administering  the  Operator  Demographics 
Questionnaire.  This  questionnaire  asked  for  the  operator’s  current  job,  qualification  (e.g.. 
meteorologist,  weather  officer,  weather  forecaster,  or  weather  observer),  education  level. 
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and  years  of  experience.  Thirty-one  operators  (eleven  DOC  and  five  DOT  meteorologists, 
two  DOD  weather  officers,  nine  DOD  weather  forecasters,  and  four  DOD  weather 
observers)  completed  this  questionnaire.  The  education  level  of  the  31  respondents  ranged 
from  high  school  through  masters  degree.  The  number  of  years  of  weather-related 
experience  ranged  from  less  than  1  year  to  over  30  years.  The  range  of  education  levels 
and  number  of  years  of  weather-related  experience  were  representative  of  each  agency’s 
personnel. 

3.3.1 .2  Operator  Questionnaire.  The  operators  answered  the  Operator  Questionnaire  (see 
paragraphs  2.1 .1.1  and  2. 1.2.1)  which  addressed,  in  addition  to  the  other  objectives, 
whether  current  qualifications  were  adequate  for  effective  NEXRAD  use.  All  operators 
assessed  qualification  requirements  for  effective  use  of  the  PUP.  In  addition,  DOC  and 
DOD  operators  assessed  qualification  requirements  for  the  effective  use  of  the  UCP. 

3.3.1. 3  Site  Supervisor  and  Agency  Specialists.  Site  supervisors  at  the  OKC  WSFO  and 
the  Tinker  AFB  BWS  and  agency  specialists  provided  the  test  team  with  position 
qualification  assessment  reports. 

3.3.2  Results  and  Conclusions. 

3.3.2.1  There  was  a  consensus  among  operators,  supervisors,  and  specialists  of  all  three 
agencies  that  current  agency  position  qualifications  were  adequate  for  NEXRAD.  However, 
they  stated  that  without  proper  training,  personnel  having  these  qualifications  will  not  be 
able  to  use  NEXRAD  PUPs  and  UCPs  effectively  for  the  duties  of  their  assigned  positions 
(e.g.,  meteorologist,  weather  officer,  weather  forecaster,  or  weather  observer).  (The 
adequacy  of  NEXRAD  operator  training  plans  are  addressed  in  objective  ES-9.) 

3.3.2.2  In  addition,  site  supervisors  and  agency  specialists  provided  additional  position 
qualification  assessment  comments.  DOD  supervisors  stated  that  weather  observers  could 
be  trained  to  perform,  without  direct  supervision,  observer-related  PUP  functions  and  to 
enter  UCP  commands  under  a  forecaster’s  guidance.  DOC  supervisors  and  specialists 
stated  that  although  current  DOC  plans  do  not  include  training  for  meteorological 
technicians  on  the  use  of  NEXRAD,  personnel  in  these  positions  could,  with  proper  training, 
assist  meteorologists  in  performing  NEXRAD  duties. 

3.3.3  Recommendation.  For  JSPO  and  users:  Ensure  effective  and  appropriate  NEXRAD 
operations  training  is  provided  to  agency  personnel  (also  see  objective  ES-9,  Training). 

3.4  OBJECTIVE  E-4.  Evaluate  NEXRAD  capability  to  provide  required  operational  support 
to  multiple  users. 

3.4.1  Method.  To  evaluate  this  objective,  the  test  team  used  a  combination  of  existing 
operational  procedures  and  proposed  new  procedures  for  the  NEXRAD  era.  In  addition 
to  three  operational  PUPs  and  the  OSF  PUP,  the  test  team  used  a  unit  loader  to  simulate 
an  operationally  representative  processing  load.  Finally,  the  test  team  analyzed 
questionnaires,  operator  comments,  product  response  times,  and  product  availability  logs. 

3.4.1. 1  Operational  Procedures.  The  test  team  operated  NEXRAD  in  accordance  with 
current  agency  operations  manuals  and  procedures  (i.e.,  WSOM,  SOPs,  etc.).  For 
NEXRAD-unique  requirements,  the  test  team  used  the  draft  Federal  Meteorological 
Handbook  No.  1 1  (FMH-1 1 )  and  agency-prepared  test  operating  procedures  (TOPs).  These 
documents  reflected  planned  agency  taskings  for  required  forecast  products  and  services. 
As  described  in  FMH-1 1,  designated  agency  representatives  from  the  three  agency  test 
sites  periodically  met  as  the  URC  to  coordinate  the  operation  of  NEXRAD. 
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3.4.1. 2  Unit  Loader  Simulation.  In  conjunction  with  the  RPG  processing  load  generated 
by  the  three  associated  PUPs  at  the  operational  sites  and  the  OSF  PUP.  a  unit  loader 
was  used  to  generate  an  additional  processing  load  which  was  representative  of  a  multiple- 
user  (19  user)  NEXRAD  site.  The  unit  loader  simulated  requests  for  products  considered 
typical  of  four  associated  PUPs,  eight  nonassociated  PUPs,  two  other  users,  and  one  RPG 
principal  user  external  system  (PUES)  port.  The  unit  loader  recorded  product  request 
information  such  as  time  of  request,  time  of  receipt,  and  product  availability  for  simulated 
users,  along  with  such  load  factors  as  number  of  users  (both  real  and  simulated),  product 
request  scenarios,  and  active  scan  strategy.  The  unit  loader  did  not  record  responsiveness 
statistics  for  the  three  operational  PUPs  and  the  OSF  PUP.  The  test  team  ran  the  unit 
loader  using  product-request  scenarios  which  were  coordinated  and  approved  by 
representatives  from  DOC,  DOD,  and  DOT.  Four  different  scenarios  were  developed  prior 
to  IOT&E(2)  and  used  throughout  the  test;  three  specified  representative  product  requests 
for  convective  and  stratiform  activity  and  a  fourth  specified  a  clear-air  scenario.  All 
scenarios  were  developed  using  a  strong  emphasis  on  requests  for  derived  products  while 
minimizing  the  requests  for  base  products. 

3.4.1 .3  Operator  Questionnaire.  The  test  team  used  the  Operator  Questionnaire  to  record 
opinions  on  NEXRAD’s  responsiveness  during  IOT&E(2)  part  B.  The  Operator 
Questionnaire  was  administered  at  the  end  of  Part  B.  The  rating  for  this  objective  was 
based  on  the  operators’  median  response  to  the  question  "What  was  the  overall 
effectiveness  of  NEXRAD  in  providing  requested  products  in  a  timely  manner  when  you 
operated  the  unit  in  various  weather  scenarios  at  the  representative  maximum  load?” 
(question  number  4,  appendix  F).  The  test  team  compared  this  median  questionnaire 
response  with  the  criterion.  The  criterion  was  a  median  rating  of  4  or  greater  on  a  6- 
point  scale  (see  table  11-2)  (ranging  from  1  *  completely  ineffective  to  6  =  completely 
effective).  A  score  of  4  or  greater  would  indicate  that  the  NEXRAD  system  provided 
requested  products  in  a  timely  manner  when  operating  at  a  representative  maximum  load. 

3.4.1. 4  Responsiveness  Questionnaire.  To  document  the  responsiveness  of  NEXRAD 
during  a  specific  operator  shift  and  unit  loader  scenario,  test  team  operators  completed 
the  Responsiveness  Questionnaire.  At  the  end  of  each  shift,  operators  used  this  form  to 
document  specific  demonstrated  responsiveness  characteristics  of  NEXRAD.  Later,  the 
test  team  deputy  for  data  management  and  analysis  annotated  on  the  Responsiveness 
Questionnaire  which  unit  loader  scenario,  if  any,  was  active  during  the  shift.  The  operators 
used  their  own  completed  and  annotated  Responsiveness  Questionnaires  as  memory 
joggers  when  answering  the  multiple-user  MOE  questions  on  the  Operator  Questionnaire. 

3.4.1. 5  Site  Supervisors  and  Agency  Specialists.  Site  supervisors,  meteorological 
specialists,  and  communications  specialists  from  the  using  agencies  provided  reports  on 
NEXRAD  responsiveness  issues.  The  test  team  used  the  comments  provided  in  these 
reports  to  supplement  and  expand  on  specific  aspects  of  NEXRAD  multiple-user  support. 

3.4.2  Results  and  Conclusions.  NEXRAD’s  capability  to  provide  required  operational 
support  to  multiple  users  met  the  users'  requirement.  The  Operator  Questionnaire  median 
response  for  31  operators  was  4  (mildly  effective,  meets  operator’s  minimum  operational 
needs).  However,  special  cases  were  identified  and  are  described  below  where  NEXRAD 
did  not  meet  specific  portions  of  this  overall  objective. 

3.4.2. 1  For  associated  users,  operators  from  the  three  test  sites  stated  that,  in  general, 
NEXRAD  provided  routine  product  set  (RPS)  products  in  a  timely  manner  including  times 
when  the  unit  was  operated  in  a  mode  simulating  a  1 9-user  configuration.  Data  from  the 
unit  loader  appeared  to  support  this  evaluation.  Overall  RPS  product  availability  for  the 
simulated  users  was  99.6  percent  for  all  weather  scenarios  run  during  Part  B  as  shown 
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in  table  111-1.  The  unit  loader  recorded  only  one  event  of  narrowband  loadshedding  for 
the  simulated  associated  user  PUPs. 


Table  111-1 

Unit  Loader  RPS  Statistics 


Operating 

Mode 

Products 

Received 

Products 

CPU 

Loadshed 

Products 

Memory 

Loadshed 

Products 

Missina 

Percent 

Unavailable 

Precipitation 

32230 

40 

13 

10 

0.2% 

Clear  Air 

4507 

57 

10 

19 

1 .9% 

Total 

36737 

97 

23 

29 

0.4% 

Note:  Unit  Loader  statistics  are  for  simulated  PUPs  only. 

CPU  Loadshed:  Products  not  received  as  the  result  of  Central  Processing  Unit  (CPU) 
loadshedding. 

Memory  Loadshed:  Products  not  received  as  the  result  of  memory  allocation  not  available. 

Missing:  Products  not  received;  not  attributed  to  CPU  or  Memory 
Loadshedding.  This  included  the  one  narrowband  loadshedding  event. 


3.4.2.2  In  contrast  to  the  operator  evaluation  and  unit  loader  statistics,  test  team 
specialists  observed  that  narrowband  loadshedding  at  the  three  operational  PUP  sites 
prevented  the  receipt  of  many  RPS  products  during  periods  of  widespread  precipitation. 
This  discrepancy  was  possibly  due  to  differences  in  the  number  and  type  of  RPS  products 
on  the  simulated  users’  RPS  lists  versus  the  actual  PUPs'  lists.  The  simulated  PUPs'  RPS 
lists,  which  remained  constant  throughout  the  test  and  were  based  on  the  coordinated 
agency  concepts  (see  paragraph  3.4.1. 2),  contained  mostly  derived  products  and  relatively 
few  base  products.  However,  during  IOT&E(2),  operators  were  allowed  to  modify  their  RPS 
lists  at  their  sites.  As  the  limitations  with  some  of  the  derived  products  became  apparent, 
operators  placed  more  base  products  and  relatively  few  derived  products  on  their  RPS  lists. 
The  base  products  had  longer  transmission  times  than  derived  products  which  increased 
the  likelihood  of  narrowband  loadshedding  for  the  actual  PUPs  versus  the  simulated  PUPs. 
Table  111-2  shows  product  transmission  times  for  derived  and  base  products  for  one 
widespread  precipitation  event. 

3.4.2.3  Operators  stated  that  while  some  one-time  requested  products  were  received  in 
a  timely  manner,  the  responsiveness  of  many  one-time  requested  products  did  not  meet 
their  needs.  Unit  loader  statistics  supported  this  statement.  These  statistics  indicated  that 
one-time  product  response  times  varied  considerably  depending  on  the  amount  of  data 
that  were  sent  over  the  narrowband  lines  and  whether  or  not  the  product  was  already 
generated  at  the  RPG.  Most  base  products  were  received  within  1  minute  from  request. 
Most  derived  products  were  received  within  30  seconds.  However,  those  products  that 
would  normally  be  requested  via  the  one-time  request  feature  (e.g.,  cross-section  products 
and  Weak-Echo  Region  (WER)  products)  had  a  mean  response  time  on  the  order  of  2.5 
to  3  minutes  during  convective  activity.  Because  of  their  reliance  on  cross-section  and 
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WER  products  during  the  test  and  dialup  feature  limitations,  the  DOT  operators  stated  that 
the  responsiveness  of  the  N EXRAD  system  did  not  meet  their  operational  requirements. 


Table  111-2 

Approximate  Product  Transmission  Times  for  a  Widespread  Precipitation  Event 

(0000  GMT-0200  GMT,  23  Jun  89) 


Base  Products 

Time  (seconds) 

Derived  Products 

Time  (seconds) 

Reflectivity  (1  km) 

23 

Severe  Weather 

less  than  1 

Reflectivity  (2  km) 

19 

Probability 

Velocity  (1  km) 

16 

Hail 

7 

Spectrum  Width  (1  km) 

23 

Mesocyclone 

1 

Velocity  (1/4  km) 

17 

Storm  Structure 

4 

Spectrum  Width  (1/4  km) 

32 

Layered  Composite 
Turbulence 

2 

3.4.2.4  For  nonassociated  users,  operators  and  specialists  identified  several  deficiencies. 
First,  dialup  procedures  for  acquiring  products  from  multiple  RPGs  were  cumbersome  and 
time-consuming.  This  deficiency  will  particularly  impact  agency  centers  requiring  routine 
access  to  multiple  RPGs.  Second,  the  RPG  telephone  number  directory  could  only  contain 
a  maximum  of  1 2  digits  for  each  RPG,  making  long-distance  dialing  through  most  facility 
switchboards  impossible.  Third,  background  maps  from  nonassociated  RPGs  were 
automatically  deleted  from  the  PUP  data  base  after  only  6  hours.  In  addition,  the 
functionality  to  store  and  retrieve  maps  using  optical  disk  media  was  inoperable.  Therefore, 
operators  had  to  repeatedly  request  maps  over  dialup  lines.  Fourth,  unit  loader  simulations 
indicated  product  receipt  times  for  nonassociated  PUPs  were  generally  50  percent  greater 
than  those  for  associated  PUPs.  These  slower  product  receipt  times  were  likely  the  result 
of  the  simulated  nonassociated  PUPs  operating  at  a  lower  narrowoand  transmission  rate. 
In  the  limited  production  phase  design,  the  transmission  rates  for  nonassociated  PUPs  are 
planned  to  be  upgraded  to  the  associated  PUPs’  transmission  rate. 

3.4.2.5  Site  supervisors  and  specialists  found  that  the  multiple  site  coordination  procedures 
used  during  IOT&E(2)  were  effective,  but  additional  issues  need  to  be  addressed.  Site 
supervisors  found  the  URC  was  an  effective  forum  for  the  principal  user  agencies  to 
coordinate  the  use  of  NEXRAD.  Test  team  specialists  noted  that  URC-developed 
agreements  need  to  be  quickly  incorporated  into  each  station’s  operating  procedures. 
Further,  the  specialists  identified  the  need  for  strong  agency  support  and  guidance 
regarding  multiple-user  support  functions  and  how  NEXRAD-related  responsibilities  relate 
to  current  duty  priorities.  The  need  for  this  guidance  was  particularly  demonstrated  when 
the  UCP  operator’s  area  of  interest  was  not  threatened  by  significant  weather  while  severe 
weather  was  entering  the  warning  area  of  a  different  associated  user. 

3.4.3  Recommendations. 

3.4.3. 1  For  JSPO: 

a.  Provide  an  effective  capability  to  acquire  products  from  multiple  RPGs.  (SRs  395. 
393,  394,  515) 


b.  Provide  the  capability  to  retain  nonassociated  background  maps  in  a  separate  PUP 
storage  area.  (SR  410) 

c.  Provide  the  capability  to  store  and  retrieve  nonassociated  RPG  background  maps. 
(SR  338,  326) 

d.  Investigate  the  adequacy  of  NEXRAD  to  support  receipt  of  products  during  peiiods 
of  widespread  precipitation.  (SR  502) 

e.  Ensure  cross-section  and  WER  products  are  received  in  a  timely  manner. 

3.4.3.2  For  users: 

a.  Develop  procedures  for  responsive  implementation  of  URC-coordinated  changes 
at  individual  associated  site  locations. 

b.  Provide  guidance  regarding  multiple-user  support  functions  and  how  NEXRAD- 
related  responsibilities  relate  to  current  duty  priorities. 

3.5  OBJECTIVE  E-5.  Evaluate  NEXRAD  as  an  effective  aid  in  preparing  accurate  and 
timely  weather  advisories. 

3.5.1  Method.  To  evaluate  this  objective,  operators  used  NEXRAD  information  to  assist 
in  preparing  operational  weather  advisories  during  IOT&E(2).  The  test  team  used  the 
Operator  Questionnaire  to  evaluate  general  NEXRAD  advisory  performance  during  IOT&E(2) 
Part  B.  In  addition,  the  test  team  collected  weather  advisory  verification  statistics  and 
specialists'  comments  to  support  the  operators'  evaluation. 

3.5.1. 1  Weather  Advisory  Procedures.  NEXRAD  was  operated  and  evaluated  using 
existing  agency  weather  advisory  procedures  as  well  as  existing  and  test-specific  weather 
advisory  support  requirements.  DOC,  DOD,  and  DOT  operators  used  NEXRAD  information 
to  assist  them  in  preparing  advisories  for  existing  criteria.  In  addition  to  the  existing  DOD 
advisory  criteria  at  Tinker  AFB  that  do  not  require  a  lead  time,  five  DOD  specialists 
prepared  test  specific  forecast  advisories  for  Tinker  AFB  that  required  a  positive  leadtime 
during  selected  shifts  in  Part  A. 

3.5.1. 2  Advisory  Verification  and  Statistics.  Using  the  same  methodology  identified  in 
paragraphs  3. 1.1. 4  and  3. 1.1. 5,  the  test  team  collected  verification  information  and 
calculated  advisory  statistics  to  assess  the  accuracy  and  timeliness  of  weather  advisories. 
Operators  made  maximum  use  of  existing  agency  verification  networks.  In  addition.  DOT 
operators  requested  pilot  reports  from  nearby  air  traffic  control  center  and  tower  facilities. 
For  the  DOD  forecast  advisories,  the  test  team  collected  and  analyzed  the  verification 
statistics  of  POD,  FAR,  and  leadtime  (see  Glossary  in  appendix  E  for  definitions). 

3.5.1 .3  Operator  Questionnaire.  The  test  team  used  the  Operator  Questionnaire  to  collect 
the  opinion  of  operators  on  NEXRAD  as  an  aid  for  weather  advisories.  The  Operator 
Questionnaire  was  administered  at  the  end  of  Part  B.  The  rating  for  this  objective  was 
based  on  the  operators’  median  response  to  the  question  "What  was  the  overall 
effectiveness  of  NEXRAD  as  an  aid  for  you  in  preparing  weather  advisories?"  (question 
number  5,  appendix  F).  The  test  team  compared  this  median  response  with  the  criterion. 
The  criterion  was  a  median  rating  of  4  or  greater  on  a  6-point  scale  (ranging  from  1  = 
completely  ineffective  to  6  =  effective).  A  score  of  4  or  greater  would  indicate  that 
NEXRAD  was  an  effective  aid  in  preparing  weather  advisories. 
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3.5. 1.4  Specialists.  Doppler  radar  specialists  from  NSSL  and  the  principal  user  agencies 
provided  reports  addressing  the  effectiveness  of  NEXRAD  as  an  aid  in  preparing  advisories. 
The  NSSL  report  focused  on  NEXRAD  algorithm  performance  while  agency  specialists 
focused  on  general  advisory  support  issues. 

3.5.2  Results  and  Conclusions.  NEXRAD  met  the  users’  requirement  for  weather 
advisories.  The  Operator  Questionnaire  median  response  for  the  27  forecasters  and 
meteorologists  from  the  three  agencies  was  4  (mildly  effective,  meets  operator’s  minimum 
operational  needs). 

3.5.2.1  Operators  stated  that  NEXRAD  met  their  minimum  operational  requirements  as 
an  aid  in  preparing  accurate  and  timely  weather  advisories.  Operators  reported  that  the 
resolution  of  the  reflectivity  products  allowed  them  to  accurately  identify  the  location  of 
significant  weather  with  respect  to  specific  advisory  and  aircraft  route  locations.  The 
sensitivity  of  the  reflectivity  products  allowed  operators  to  identify  many  features  such  as 
gust  fronts,  thunderstorm  outflow  boundaries,  and  fine  lines.  Identification  of  these  features, 
combined  with  the  use  of  Velocity  Azimuth  Display  (VAD)  wind  profiles  and  base  velocity 
products  to  identify  inversions  and  low-level  jet  streams,  enabled  operators  to  provide  timely 
terminal  wind  advisories  and  low-level  wind  shear  advisories.  In  addition,  during  Part  A. 
DOD  specialists  achieved  a  POD  of  100  percent  and  a  FAR  of  17  percent  for  the  seven 
prepared  forecast  advisories. 

3.5.2.2  However,  the  test  team  identified  deficiencies  that  impacted  the  operational 
effectiveness  of  NEXRAD  for  weather  advisory  support.  Operators  did  not  find  useful 
information  in  the  layered-turbulence  products.  This  limitation  prevented  DOT  from  using 
this  product  in  preparing  aircraft  advisories  for  turbulence,  a  key  element  of  required  CWSU 
weather  support.  Therefore,  DOT  operators  used  reflectivity-based  products  to  infer 
potential  turbulent  regions  associated  with  convective  activity  and  velocity -based  products 
to  infer  potential  turbulent  regions  in  clear  air  conditions.  Previously  identified  deficiencies 
with  velocity  dealiasing  and  range-folded  velocity  data  often  prevented  operators  from 
determining  the  strength  of  winds  associated  with  convective-related  features  identified  in 
the  reflectivity  data  (e.g„  gust  fronts).  When  convective  activity  was  within  10  nm  of  the 
RDA.  the  delivered  scan  strategies  did  not  provide  the  operators  with  an  adequate  vertical 
distribution  of  the  storm’s  reflectivity.  This  difficulty  was  due  to  the  delivered  scan  strategy 
only  sampling  below  20  degrees  in  elevation.  Therefore,  operators  could  not  see  the 
upper  levels  of  the  storm.  As  a  result,  DOD  operators  had  difficulties  in  canceling 
observed-thunderstorm  advisories  for  Tinker  AFB  when  convective  activity  was  within  10  nm 
of  the  RDA.  The  test  team  noted  two  limitations  associated  with  the  use  of  the  automated 
alert  feature  that  reduced  this  feature’s  effectiveness.  First,  the  current  state  of  the  storm- 
series  algorithms  appeared  to  produce  frequent  false  indications  of  significant  weather  (e  g 
hail  and  mesocyclonic  shear).  NSSL  specialists  estimated  that  greater  than  50  percent 
of  the  mesocyclone  alerts  were  false.  Second,  specialists  observed  that  because  of 
inadequate  applications  training,  operators  did  not  always  know  how  to  apply  alert 
thresholds  and  alert  areas  to  match  existing  meteorological  conditions. 

3.5.3  Recommendations. 

3.5.3. 1  For  JSPO: 

a.  Provide  effective  layered  turbulence  products.  (SRs  160,  421) 

b.  Eliminate  the  impact  of  range-folded  data  on  the  velocity-based  products. 
(SR  219) 

c.  Provide  an  effective  velocity  dealiasing  algorithm.  (SRs  500,  062B,  411) 
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d.  Provide  effective  hail  and  mesocyclone  algorithms.  (SRs  380,  228A) 

3.5.3. 2  For  JSPO  and  users:  Provide  adequate  training  on  the  appropriate  application 
of  the  automated  alert  feature  for  each  users'  weather  support  requirements.  (SRs  015. 
247,  455) 

3.6  OBJECTIVE  E-6.  Evaluate  NEXRAD  as  an  effective  aid  in  providing  routine  weather 
services. 

3.6.1  Method.  To  evaluate  this  objective,  operators  used  NEXRAD  information  to  assist 
in  providing  routine  weather  services  during  IOT&E(2).  The  test  team  used  the  Operator 
Questionnaire  to  evaluate  general  NEXRAD  routine  weather  support  performance  during 
IOT&E(2)  Part  B.  In  addition  to  these  current  weather  services,  DOC  operators  edited 
RCMs  during  selected  shifts. 

3.6. 1.1  Operating  Procedures.  Test  team  operators  provided  routine  meteorological 
services  as  outlined  in  the  respective  agencies'  current  operating  procedures.  These 
routine  services  included  terminal  forecasts  (DOD),  surface  weather  and  radar  observations 
(DOD),  weather  briefings  (DOC,  DOD,  DOT),  nowcasts  (DOT,  DOC),  and  routine  weather 
forecasts  and  statements  (DOC). 

3.6.1. 2  Operator  Questionnaire.  The  test  team  used  the  Operator  Questionnaires  to 
evaluate  NEXRAD  as  an  aid  in  providing  routine  weather  support.  The  Operator 
Questionnaire  was  administered  at  the  end  of  Part  B.  The  ratings  for  this  objective  were 
based  on  the  operators’  median  responses  to  the  following  questions:  (a)  "What  was  the 
overall  effectiveness  of  NEXRAD  as  an  aid  for  you  in  preparing  short-range  forecasts?" 
(b)  "What  was  the  overall  effectiveness  of  NEXRAD  as  an  aid  for  you  in  taking  surface 
weather  observations?"  (c)  "What  was  the  overall  effectiveness  of  NEXRAD  as  an  aid  for 
you  in  preparing  and  presenting  weather  briefings?"  (d)  "What  was  the  effectiveness  of 
NEXRAD  as  an  aid  for  you  in  briefing  traffic  management  on  weather  problems  that  could 
impact  local  traffic  flow  or  local  air  traffic  control  capabilities?"  (questions  numbered  6 
through  9  in  appendix  F).  The  test  team  compared  these  four  median  responses  with 
the  corresponding  criteria.  The  criterion  for  each  of  the  four  aspects  of  routine  services 
was  a  median  rating  of  4  or  greater  on  a  6-point  scale  (ranging  from  1  =  completely 
ineffective  to  6  =  completely  effective).  A  score  of  4  or  greater  would  indicate  that 
NEXRAD  was  an  effective  aid  in  providing  that  aspect  of  routine  support. 

3.6. 1.3  Radar  Coded  Message  (RCM).  The  test  team  assessed  the  impact  of  the  new 
DOC  requirement  of  editing  and  transmitting  an  RCM  each  hour.  During  selected  shifts 
in  IOT&E(2)  Part  B.  DOC  operators  edited  the  RCM  using  Guidance  in  the  draft  FMN-i  i. 
Part  E.  Under  various  weather  situations,  the  test  team  collected  RCM  editing  times  and 
the  percentage  of  RCMs  edited  before  dissemination. 

3.6.2  Results  and  Conclusions.  NEXRAD  met  the  users’  minimum  requirement  as  an 
effective  aid  in  short-range  forecasts,  surface  observations,  briefings,  and  aircraft  traffic 
management.  The  median  response  from  the  31  operators  for  forecasts,  surface 
observations,  and  flow  control  was  a  4  (mildly  effective,  meets  operator’s  minimum  needs). 
The  median  response  for  26  meteorologists  and  forecasters  for  weather  briefings  was  a 
4.5  (between  mildly  and  highly  effective). 

3.6.2. 1  NEXRAD  was  an  effective  aid  in  the  preparation  of  short  range  (0  to  6  hour) 
forecasts.  The  high  resolution  and  sensitivity  of  NEXRAD  aided  in  the  identification  of 
fronts,  wind  shift  lines,  precipitation  areas,  and  dry  lines.  Clear-air  mode  operation  was 
particularly  effective  in  identifying  small-scale  features.  VAD  and  base  velocity  products. 
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when  not  contaminated  by  large  areas  of  range-folded  and  incorrectly  dealiased  data,  aided 
in  the  preparation  of  surface  forecasts  and  in  diagnosing  vertical  wind  field  changes. 


3.6. 2.2  DOD  observers  stated  that  NEXRAD  was  an  effective  aid  in  preparing  surface 
weather  observations.  Observers  used  the  Echo  Tops  product,  Storm  Tracking  Information 
product,  and  time-lapse  feature  to  determine  storm  identification,  location,  and  movement 
for  inclusion  in  surface  observation  remarks.  Based  on  current  requirements,  DOD 
forecasters  and  observers  stated  they  were  able  to  prepare  a  reflectivity-only  radar 
observation  more  accurately  and  typically  in  less  than  half  the  time  with  NEXRAD  than 
is  presently  required  for  the  FPS-77  weather  radar. 

3.6.2.3  Operators  stated  that  NEXRAD  was  an  effective  aid  in  preparing  and  presenting 
weather  briefings.  DOD  forecasters  stated  that  the  ability  to  time-lapse  color  radar 
information  and  to  remote  that  information  to  the  briefing  counter  was  particularly  valuable. 
DOC  operators  were  able  to  prepare  civil  defense  briefings  using  NEXRAD  primarily 
because  of  the  detailed  reflectivity  data  placement  on  the  county  and  operator-generated 
city  background  maps.  DOT  and  DOD  operators  noted  the  usefulness  of  the  reflectivity 
and  VAD  products  aided  in  displaying  the  meteorological  conditions  for  planned  briefings. 
However,  the  DOT  operators  stated  that  on-demand  briefing  effectiveness  was  degraded 
because  one-time  and  dialup  product  requests  were  not  responsive  (see  objective  E-4). 

3.6.2.4  The  automatic  scan  mode  deselection  feature  often  forced  an  operationally 
undesirable  switch  to  the  precipitation  mode  because  of  anomalous  propagation  (AP).  The 
design  of  the  deselection  feature  prevented  the  operator  from  reselecting  the  clear-air  mode 
for  at  least  1  hour  after  changeover.  At  these  times,  the  increased  detection  capability 
of  the  clear-air  mode  was  not  available  to  support  routine  operations. 

3.6.2.5  All  DOC  operators  stated  that  the  new  requirement  of  editing  the  RCM  produced 
a  significant  increase  in  their  workload.  Operators  spent  significant  time  verifying  and 
removing  residual  clutter,  AP,  and  false  indications  of  mesocyclones  and  hail.  The 
automated  remarks  in  Part  C  of  the  RCM  did  not  provide  useful  information  and  required 
extensive  editing.  Operators  stated  that  because  of  other  mission  requirements.  42  RCMs 
were  sent  out  without  being  edited.  19  of  which  were  during  severe  weather  (see  table 
111-3). 

3.6.3  Recommendations. 

3.6.3. 1  For  JSPO: 

a.  Eliminate  the  impact  of  range-folded  data  on  the  velocity-based  products. 
(SR  219) 

b.  Provide  an  effective  velocity  dealiasing  algorithm.  (SRs  500,  082B,  441) 

c.  Reduce  the  impact  of  the  RCM  on  operator  workload.  (SRs  484.  258A.  307,  358. 
411,  333,  427,  336,  385) 

d.  Ensure  one-time  products  are  received  in  a  timely  manner  for  on-demand  briefings. 
(SR  166  A) 

e.  Provide  an  effective  capability  to  acquire  products  from  multiple  RPGs.  (SRs  395. 
393.  394,  515) 
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Table  111-3 


RCM  Editing  Distributions 


Number  of  RCMs 

Percentaqe  of 

Total  RCMs  Required 

176 

100 

Sent  Out  Edited 

Appropriate 

100 

57 

Inappropriate 

3 

2 

Total 

103 

59 

Sent  Out  Unedited 

Reason: 

Other  mission  requirements 

19 

11 

Severe  Weather 

Other  mission  requirements 

23 

13 

Nonsevere  Weather 

Total 

42 

24 

Not  Sent  Out 

Reason: 

Radar  Down 

31 

17 

Total 

31 

17 

3.6.3.2  For  JSPO  and  users:  Provide  the  UCP  operator  the  capability  to  override  the 
automatic  scan  mode  deselect  feature  and  1-hour  timeout  when  operationally  required. 
Ensure  the  FMH-11  allows  the  UCP  operator  to  use  this  capability.  (SR  250A) 

3.7  OBJECTIVE  E-7.  Assess  NEXRAD  as  an  effective  aid  to  meeting  agency  mission 
requirements  when  changing  to,  operating  on,  and  recovering  from  backup  power. 

3.7.1  Method.  The  test  team  assessed  the  performance  of  NEXRAD  when  operating  on 
and  transitioning  to  and  from  backup  power.  The  test  team  used  the  Operator 
Questionnaire  results  and  the  maintenance  logs  to  assess  this  objective. 

3.7. 1.1  Operational  Procedures.  Operators  used  NEXRAD  as  an  aid  in  conducting 
required  weather  support  services.  Current  agency  weather  support  plans,  used  by  the 
test  team,  required  weather  services  to  continue  following  the  loss  of  commercial  power. 
In  addition  to  the  unplanned  loss  of  commercial  power,  operators  often  switched  to  backup 
power  in  anticipation  of  commercial  power  fluctuations  during  severe  weather. 

3.7.1 .2  Operator  Questionnaire.  The  test  team  used  the  Operator  Questionnaire  to  record 
the  opinion  of  the  operators  on  NEXRAD's  effectiveness  during  power  transitions  and  while 
operating  on  backup  power.  The  Operator  Questionnaire  was  administered  to  31  test  team 
operators  at  the  end  of  Part  B.  The  Operator  Questionnaire  requested  comments 
addressing  the  quality,  continuity,  and  availability  of  NEXRAD  products  while  operating  on 
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backup  power  and  following  transitions  to  and  from  backup  power.  The  operators  also 
provided  comments  addressing  any  significant  NEXRAD-miated  workload  impacts  caused 
by  backup  power  transition  recovery  actions. 

3.7. 1.3  Maintenance  Data  Review.  Maintenance  technicians  noted  reliability  and 
maintainability  problems  caused  by  power  transitions  or  the  use  of  backup  power.  They 
documented  equipment  deficiencies  as  service  reports.  Outage  times  were  collected  on 
the  MDC  forms. 

3.7.2  Results  and  Conclusions.  NEXRAD  was  not  an  effective  aid  in  meeting  agency 
mission  requirements  when  changing  to  and  recovering  from  backup  power. 

3.7.2.1  During  IOT&E(2)  Part  B,  the  system  failed  17  times  (RDA  4  times,  RPG  12  times, 
and  PUP  1  time)  because  of  power  transitions-whether  unscheduled  or  operator-initiated. 
In  these  cases,  a  maintenance  action  and  a  manual  restart  were  required.  Outage  times 
resulting  from  power  transfers  ranged  from  11  minutes  to  8  hours  54  minutes.  These 
failures  resulted  in  an  increase  in  workload,  an  increase  in  maintenance  interventions,  and 
the  loss  of  critical  radar  data.  Operators  stated  that  the  loss  of  critical  radar  data  during 
significant  weather  situates  resulted  in  a  significant  decrease  in  the  effectiveness  of 
NEXRAD  as  an  aid  in  providing  weather  warning  and  advisory  support.  Conversely, 
operators  observed  that  the  three  operational  PUPs  recovered  automatically  after  power 
transitions  except  for  one  event  at  Tinker  AFB. 

3.7.P.2  Operators  did  not  observe  any  change  in  system  performance  or  operator  workload 
when  NEXRAD  was  operating  on  backup  power. 

3.7.3  Recommendation.  For  JSPO:  Ensure  the  RDA  and  RPG  effectively  and 
automatically  return  to  an  operational  state  following  power  transitions.  (SRs  317,  087B) 

3.8  OBJECTIVE  E-8.  Assess  NEXRAD  electromagnetic  compatibility  (EMC). 

3.8.1  Method.  The  test  team  maintenance  technicians  and  operators  noted,  by  exception, 
apparent  EMC  problems  that  produced  performance  anomalies  in  NEXRAD  or  in  nearby 
electronic  systems.  They  assessed,  where  possible,  anomalies  that  may  have  been 
attributable  to  EMC  problems. 

3.8.2  Results  and  Conclusions.  The  test  team  maintenance  technicians  noted  one 
apparent  EMC  problem--a  wavy  presentation  on  the  RDA  applications  terminal  throughout 
IOT&E(2).  Although  the  technicians  replaced  the  monitor,  the  wavy  presentation  continued. 
Operators  did  not  observe  any  EMC  incidents  associated  with  the  operation  of  NEXRAD 
equipment,  nor  was  there  any  observable  effect  on  any  nearby  equipment. 

3.8.3  Recommendation.  For  JSPO:  Investigate  and  resolve  the  cause  of  the  RDA 
applications  terminal  having  a  wavy  presentation.  (SRs  051  A,  131) 

3.9  OBJECTIVE  ES-9.  Assess  the  adequacy  of  the  planned  NEXRAD  training  to  provide 
the  skills  required  to  effectively  use  and  maintain  NEXRAD. 

3.9.1  Method.  Operations,  maintenance,  and  software  personnel  received  training  for 
IOT&E(2). 

3.9. 1.1  Operations  Training.  Operations  training  was  a  government-designed,  six-phase 
training  course  specially  developed  for  IOT&E(2).  Five  of  the  six  phases  of  training  were 
government  provided.  The  fourth  phase  of  operations  training  was  a  2-week  contractor- 
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provided  course  which  was  not  representative  of  the  government  planned  4-week  operator 
course. 

3.9.1. 2  Maintenance  Training.  The  maintenance  training  course  for  IOT&E(2)  was  initially 
designed  to  be  the  same  as  the  planned  6-week  maintenance  course.  However,  based 
on  the  results  of  the  first  half  of  the  course,  the  JSPO  instructed  the  contractor  to 
restructure  the  course  and  to  add  another  week.  At  the  conclusion  of  the  7  weeks  of 
training,  the  JSPO  discovered  that  crucial  sections  of  the  course  had  not  been  provided. 
The  test  team  technicians  received  an  additional  3-day  training  course  in  the  middle  of 
IOT&E(2)  Part  A  to  correct  this  deficiency. 

3.9.1. 3  Software  Training.  Software  training  was  a  7-week,  contractor-provided  course 
that  was  intended  to  be  an  equivalent  subset  of  the  planned  14-week  software  course. 
In  the  IOT&E(2)  course,  only  four  CPCIs  were  addressed  in  detail  rather  than  the  18  CPCIs 
to  be  presented  in  the  14-week  course. 

3.9.1. 4  Data  Analysis.  Based  on  test  team  training  specialists’  review  of  the  contractor’s 
training  plans,  course  outlines,  and  training  aids  together  with  knowledge  gained  through 
classroom  and  hands-on  training  and  the  results  of  questionnaires,  the  test  team  assessed 
the  adequacy  of  planned  NEXRAD  operations,  maintenance,  and  software  training.  In 
addition,  operations  training  specialists  reviewed  the  comments  provided  by  the  31 
operators  from  all  three  agencies  to  assess  planned  operations  training. 

3.9.2  Results  and  Conclusions. 

3.9.2.1  Operations.  Ten  agency  training  specialists  identified  numerous  deficiencies 
associated  with  the  planned  NEXRAD  operations  courses. 

a.  Test  team  training  specialists  stated  that  the  planned  Cadre  and  Interim  Operations 
Courses  were  deficient.  The  Cadre  course  lacked  sufficient  detail  and  would  not 
adequately  prepare  agency  instructors  to  teach  NEXRAD  operations.  In  addition,  they 
stated  the  Interim  Operations  Course  would  not  support  the  training  of  students  to  the 
agency-required  skill  level.  Considering  meteorological  content,  both  courses  had 
inadequate  depth  both  in  product  interpretation  and  in  the  application  of  products  to 
different  weather  scenarios.  Further,  the  courses'  structural  deficiencies  included 
inadequate  student  and  instructor  guides.  The  order  of  presentation  of  the  topics  for  these 
courses  was  difficult  to  follow  and  not  logical. 

b.  DOC  training  specialists  stated  that  DOC/DOT-planned  computer-based  training 
(CBT)  was  an  area  of  high  risk.  Training  specialists  identified  four  significant  deficiencies 
associated  with  the  planned  CBT.  First,  the  CBT  design  did  not  include  functionality  critical 
to  NEXRAD  operations  (e.g.,  RPS  list  management,  one-time  product  requests,  and 
effective  time  lapse).  Second,  CBT  workstations  will  not  provide  the  necessary  hands- 
on  experience  required  for  confident  decision  making.  Third,  modifications  to  the  CBT 
course,  necessary  to  reflect  the  expected  changes  in  NEXRAD  functionality,  will  likely  be 
time-consuming  and  expensive.  The  time  to  modify  CBT  software  and  firmware,  test,  and 
reinstall  may  impact  training  schedules.  Fourth,  current  plans  did  not  address  required 
on-site  training. 

c.  A  comprehensive  DOD  NEXRAD  operations  training  plan  had  not  been  prepared. 
DOD  training  specialists  stated  that  general  concepts  and  unofficial  course  outlines  were 
available,  but  the  level  of  detail  contained  in  these  documents  was  inadequate  to  ensure 
that  the  training  will  prepare  operators  to  meet  mission  requirements.  The  outlined  DOD 
concept  for  NEXRAD  installation  training,  which  included  precursor,  mobile  training  team. 
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and  on-the-job  training  (OJT),  lacked  sufficient  detail  to  indicate  the  contribution  of  each 
training  phase  towards  certification.  For  instance,  skill  levels  and  prerequisite  training  for 
each  phase  were  not  part  of  an  integrated  plan.  Several  other  deficiencies  were  also 
noted,  including  the  absence  of  training  on  UCP  operations,  extended  adaptation  data 
modification,  and  system  console  operation.  Formal  course  requirements  had  not  been 
finalized;  consequently,  manpower  requirements  to  support  training  had  not  been  adequately 
defined. 

d.  The  PUP  training  mode  and  NEXRAD  archive  functionality  demonstrated  a  potential 
to  support  hands-on  operations  training.  However,  test  team-identified  deficiencies  with 
these  features  limited  tneir  usefulness  during  IOT&E(2).  For  example,  the  archive 
functionality  was  unreliable  and  the  training  mode  did  not  permit  the  operator  to  specify 
scenario  start  times. 

3.9.2.2  Maintenance.  Without  significant  changes,  the  planned  NEXRAD  maintenance 
training  will  not  provide  the  necessary  training  for  an  agency  technician  to  acquire  the 
needed  skills  to  effectively  maintain  a  NEXRAD  system  in  accordance  with  the  maintenance 
concept. 


a.  The  test  team  identified  several  deficiencies  with  planned  maintenance  training. 
First,  the  course  objectives  were  not  sufficiently  specific  to  determine  the  adequacy  of  the 
course  length.  Second,  the  course  contained  insufficient  instruction  in  several  areas, 
including  the  basic  theory  of  computer  architecture,  digital  electronics,  modems,  fiber  optics, 
communication  theory  (narrowband  and  wideband),  the  use  and  configuration  of  complex 
test  equipment,  and  software  functions  and  interfaces.  Third,  the  precursor  training,  an 
integral  part  of  the  overall  training,  was  not  addressed  in  the  contractor’s  plan.  The 
government  was  developing  a  precursor  package,  but  this  planning  was  not  complete. 
Fourth,  specialists  expressed  concern  that  the  training  course  development,  as  well  as 
hands-on  training,  might  suffer  because  of  system  time-sharing  at  the  OSF  between 
operations,  maintenance,  and  software  training;  software  development;  DT&E;  field  support; 
and  downtime  for  failures. 

b.  The  test  team  documented  several  deficiencies  associated  with  the  7-week 
IOT&E(2)  maintenance  course.  The  PTM,  which  was  used  as  the  primary  course 
reference,  was  ineffective  as  a  training  tool  (see  objective  S-15).  Training  was  presented 
without  sufficient  detail  and  did  not  interrelate  the  functionality  of  components.  The  flow 
of  instruction  did  not  follow  an  organized,  logical  plan.  The  instructors  did  not  demonstrate 
an  in-depth  knowledge  of  the  NEXRAD  system.  The  allotted  hands-on  time  did  not  achieve 
the  training  objectives.  As  a  result,  maintenance  technicians  stated  that  on  83  out  of  1 59 
maintenance  actions  documented  on  maintenance  incident  questionnaires  during  IOT&E(2). 
required  training  was  either  inadequate  (41  actions)  or  not  provided  (42  actions).  These 
training  deficiencies  directly  contributed  to  the  excessive  troubleshooting  and  repair  times 
experienced  during  IOT&E(2)  (see  objective  S-13). 

c.  The  8  hours  of  computer  maintenance  training  taught  by  Concurrent  Computer 
Corporation  personnel  during  the  3-day  additional  training  course  more  closely  achieved 
the  training  goals  because  these  lessons  were  logically  structured  and  well-presented. 
However,  the  technicians  stated  that  too  much  information  was  presented  in  too  short  a 
period. 

d.  Unless  these  training  problems  are  resolved  before  the  start  of  cadre  training,  the 
goals  of  the  NEXRAD  maintenance  concept  will  not  be  achieved  and  operational  availability 
will  probably  be  adversely  affected. 
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3.9.2.3  Software.  The  planned  NEXRAD  software  training  will  probably  not  provide  the 
skills  necessary  to  effectively  maintain  the  NEXRAD  software. 

a.  The  proposed  14-week  course  was  planned  to  be  presented  only  once  during  the 
lifetime  of  NEXRAD.  A  review  of  this  planned  course  showed  that  the  same  deficiencies 
identified  in  the  7-week  IOT&E(2)  course  (discussed  below)  will  likely  be  repeated, 
particularly  in  the  areas  of  course  structure,  level  of  detail,  and  laboratory  instruction 
techniques.  In  addition,  no  follow-on,  OJT,  or  additional  formal  training  was  planned.  No 
provisions  had  been  made  to  train  personnel  hired  after  the  course  was  presented. 

b.  The  software  evaluators  identified  several  deficiencies  with  the  7-week  IOT&E(2) 
software  maintenance  course.  First,  the  structure  of  the  course  did  not  follow  an 
organized,  logical  plan.  Class  members  were  required  to  team  information  on  their  own 
to  complete  laboratory  exercises,  only  to  receive  the  corresponding  formal  instruction  later. 
Information  about  CPCIs  was  intermixed  with  other  CPCIs  and  taught  over  several  days, 
making  it  difficult  for  class  members  to  discern  the  separate  functionality  of  each  CPCI. 
Second,  the  focus  and  level  of  detail  presented  were  not  adequate  to  maintain  the 
NEXRAD  software.  The  course  provided  an  adequate  knowledge  of  the  organization  and 
operation  of  NEXRAD  software  but  not  the  detailed  skills  and  procedures  needed  for 
software  maintenance.  Some  objectives  which  required  detailed  discussions,  such  as 
software  debug  tools,  were  presented  in  a  few  hours.  Other  objectives  which  required  less 
detail,  such  as  the  NEXRAD  overview,  took  almost  4  days.  Also,  time  was  inefficiently 
spent  going  over  each  possible  response  in  each  menu  during  class  and  in  the  laboratory. 
The  visual  aids  and  the  three  volumes  of  student  training  material  used  in  the  course 
contained  insufficient  useful  information.  Third,  there  was  insufficient  hands-on  laboratory 
experience  with  the  NEXRAD  software  maintenance  procedures.  Laboratory  time  consisted 
of  "follow-me"  exercises  rather  than  the  complete  software  problem  resolution  process. 
The  laboratories  were  not  long  enough  for  troubleshooting  software  problems.  Insufficient 
time  was  allocated  to  complete  a  build  of  a  CPCI.  The  use  of  support  tools  to  modify, 
build,  and  test  the  software  was  not  adequately  addressed. 

c.  The  course  instructors  provided  by  the  contractor  were,  however,  highly  qualified. 
They  were  knowledgeable  on  the  subject  material  and  covered  the  material  in  the  course 
outline.  System  perspectives  were  well-presented  and  gave  a  good  understanding  of  the 
NEXRAD  system  components. 

d.  Unless  the  deficiencies  identified  above  are  corrected,  software  maintainers  will 
likely  require  an  extensive  on-the-job  trial-and-error  process  to  acquire  the  skills  needed 
to  maintain  the  NEXRAD  software. 

3.9.3  Recommendations. 

3.9.3. 1  For  JSPO: 

a.  Operations: 

(1)  Ensure  adequacy  of  Personnel  Requirements,  Training,  and  Training 
Equipment  Plan  (CDRL  218)  in  meeting  agency  operations  training  requirements. 

(2)  Correct  deficiencies  associated  with  the  PUP  training  mode.  (SRs  059.  162. 
460,  579,  505) 

(3)  Correct  deficiencies  associated  with  NEXRAD  archive  functionality  to  help 
support  operator  training.  (SRs  120,  325,  351,  194,  338) 
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b.  Maintenance:  Ensure  the  technical  manuals  are  sufficiently  upgraded  and 
adequate  course  material  is  developed  to  meet  both  the  theory  and  hands-on  training 
requirements  of  the  cadre  training  course  and  the  first  increment  of  field  maintainers.  (SRs 
014,  265A,  129,  440,  138A,  227,  328,  384,  138,  286,  420,  544,  251) 

c.  Software: 

(1)  Ensure  the  contractor’s  14-week  software  maintenance  course  is  restructured 
to  follow  a  logical,  organized  plan. 

(2)  Ensure  the  focus  and  level  of  detail  of  the  contractor’s  14-week  software 
maintenance  course  provide  the  students  instruction  in  the  proper  use  of  software  tools 
necessary  to  maintain  the  NEXRAD  software. 

(3)  Ensure  the  contractor's  14-week  software  maintenance  course  provides 
adequate  hands-on  laboratory  time. 

3.9.3.2  For  JSPO  and  Users  -  Maintenance: 

a.  Ensure  the  contractor’s  maintenance  instructors  are  sufficiently  knowledgeable  of 
NEXRAD  to  teach  both  theory  and  hands-on  maintenance  for  all  functional  areas.  Until 
Unisys  demonstrates  the  ability  to  provide  an  adequate  training  course,  make  maximum 
use  of  subcontractor  equipment  training  experts  (e.g.,  Concurrent  Computer  Corporation 
training  instructors). 

b.  Ensure  detailed  lesson  plans  are  developed  well  in  advance  of  cadre  training. 
Inspect  these  plans  to  determine  adequacy  of  course  content  and  length. 

c.  Ensure  the  course  contains  an  introduction  to  all  areas  of  instruction  that  have 
not  been  previously  taught  to  current  5-ievel  technicians  (e.g.,  fiber  optics,  computer 
architecture,  etc.). 

3.9.3.3  For  DOC  and  DOT  -  Operations: 

a.  Evaluate  the  potential  of  supplementing  CBT  instruction  at  the  training  site  with 
hands-on  use  of  PUPs  and  an  RPG  using  Archive  II  playback  capability. 

b.  Prepare  training  materials  to  address  on-site,  follow-on  NEXRAD  training. 

3.9.3.4  For  DOD  -  Operations: 

a.  Ensure  a  comprehensive,  coordinated  training  plan  is  developed. 

b.  Ensure  manpower  requirements  to  meet  training  needs  are  adequately  defined 
and  personnel  are  available  in  time  to  prepare  for  cadre  training. 

3.9.3.5  For  the  OSF: 

a.  Ensure  adequate  OJT  materials  and  a  follow-on  software  maintenance  course  are 
developed  for  training  OSF  software  personnel. 

b.  Ensure  adequate  system  time  is  provided  for  operations,  maintenance,  and 
software  course  development  and  for  hands-on  instruction  during  laboratory  sessions. 
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3.10  OBJECTIVE  ES-10.  Assess  impacts  of  any  safety  hazards  associated  with  NEXRAD. 

3.10.1  Method.  The  intent  of  this  objective  was  to  identify  and,  where  possible,  eliminate 
all  safety  hazards.  Prior  to  the  start  of  IOT&E(2),  a  safety  specialist  from  Headquarters 
AFOTEC  conducted  an  on-site  safety  inspection  of  the  NEXRAD  unit  and  the  test  facility. 
Throughout  the  test,  all  test  team  personnel  were  tasked  with  assessing  and  documenting 
potential  hazards  noted  with  equipment,  operations,  and  maintenance  actions.  System 
safety  was  addressed  in  conjunction  with  ail  objectives  to  identify  potential  problem  areas 
which  may  require  future  engineering,  design  changes,  or  procedural  modifications.  Areas 
were  identified  that  may  cause  injury  to  personnel  and/or  damage  to  equipment  and  reduce 
the  effectiveness  and/or  suitability  of  the  NEXRAD  system. 

3.10.1.1  Test  team  personnel  examined  the  contractor’s  facility  plans,  drawings,  PTM,  and 
the  AFOTEC  pre-!OT&E(2)  safety  inspection  report.  All  identified  safety  hazards  were 
documented  as  SRs. 

3.10.2  Results  and  Conclusions.  The  test  team  identified  and  documented  56  safety 
deficiencies  during  IOT&E(2).  Nine  of  the  deficiencies  were  hazards  that  had  the  potential 
to  cause  death,  severe  injury,  or  major  system  damage  (Category  I).  The  immediate 
hazards  associated  with  these  nine  deficiencies  were  temporarily  resolved  for  the  test. 
However,  permanent  solutions  must  be  incorporated  into  the  production  systems.  Of  the 
remaining  47  safety  deficiencies  (Category  II),  16  had  the  potential  to  cause  minor  injury 
to  personnel,  22  had  the  potential  to  cause  either  minor  injury  to  personnel  or  minor 
damage  to  equipment,  while  the  other  9  had  the  potential  to  cause  minor  equipment 
damage  only. 

3.10.2.1  The  areas  with  the  largest  number  of  safety  deficiencies  identified  during 
IOT&E(2)  were  in  the  RDA  shelter  (16  SRs),  the  radome/tower  (15  SRs).  and  the  generator 
shelter  (6  SRs).  Additionally,  the  inadequate  warnings  and  equipment  power-down/power- 
up  procedures  in  the  PTM  produced  potentially  significant  hazards  to  personnel  and 
equipment. 

3.10.2.2  Of  the  33  safety-related  deficiencies  identified  during  IOT&E(1A)  and  IOT&Ei'IBI, 
the  test  team  revalidated  7.  Four  of  these  deficiencies  were  in  the  redesigned 
radome/tower  area. 

3.10.2.3  The  greatest  potential  for  personnel  injury  existed  within  the  radome  area.  Two 
serious  safety  deficiencies  with  the  radome  maintenance  hatch  were  identified  during 
IOT&E(1A),  and  the  same  deficiencies,  along  with  hazards  associated  with  the  radome 
davit,  were  noted  during  IOT&E(2).  The  temporary  solution  to  these  problems  was  that 
organizational-level  maintenance  personnel  would  not  be  required  to  use  the  hatch  or  davit: 
however,  a  long-term  solution  is  needed.  Additionally,  four  potentially  serious  safety 
hazards  associated  with  access  to  the  top  of  the  antenna  pedestal  and  the  radome 
obstruction  lights  were  identified  during  IOT&E(2).  First,  no  means  was  provided  to  safely 
transport  equipment/tools  to  and  from  the  top  of  the  pedestal  (hand-carrying  items  up  a 
temporary  ladder  was  prohibited  by  Military  Standard  1472C).  Second,  procedures  did  not 
require  the  use  of  a  safety  belt  while  standing  on  the  temporary  ladder  and  working.  Third, 
the  transition  between  the  temporary  ladder  and  the  fixed  ladder  on  the  back  of  the 
antenna  was  dangerous.  Fourth,  after  ascending  the  fixed  ladder  to  the  top  of  the  antenna, 
maintenance  technicians  could  not  safely  access  the  obstruction  lights.  Unless  the  proper 
safety  equipment  is  installed  and  safe  procedures  are  documented  for  working  on.  or  near, 
the  top  of  the  pedestal  for  the  production  systems,  a  high  potential  for  serious  injury  exists. 
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3.10.3  Recommendations.  For  JSPO: 

a.  Ensure  the  contractor  corrects  alt  identified  Category  I  safety  deficiencies. 

(SRs  168,  012,  010,  264A,  262A,  190,  189,  Oil,  009,  076,  049,  061  A) 

b.  Ensure  the  contractor  corrects  all  identified  Category  II  safety  deficiencies. 

(SRs  391,  463,  286,  207,  133,  404,  169,  357,  533,  032,  098,  113) 

c.  Ensure  safety  warnings  and  safe  equipment  power-down/power-up  instructions 
are  incorporated  into  all  applicable  maintenance  procedures  in  the  technical  data  (SRs 
16G,  286,  285). 

3.11  OBJECTIVE  ES-11.  Assess  factors  impacting  the  interoperability  of  NEXRAD  with 
existing  and  planned  systems. 

3.11.1  Method.  Six  test  team  specialists  and  six  software  evaluators  reviewed  contractor 
technical  documents  to  assess  the  capability  of  meeting  existing  and  future  interoperability 
requirements.  Contractor  interface  control  documents  were  compared  to  corresponding 
standards  with  respect  to  accuracy  and  level  of  detail.  The  test  team  documented 
identified  deficiencies  as  service  reports  and  assessed  the  impact  of  the  deficiencies  on 
NEXRAD's  interoperability  with  existing  and  planned  systems.  In  addition,  the  test  team 
addressed  NEXRAD’s  ability  to  manually  interoperate  (nonelectronically)  with  existing 
weather  information  systems/networks. 

3.11.2  Results  and  Conclusions. 

3.11.2.1  The  test  team  found  there  was  inadequate  information  in  the  interface  control 
documents  to  interface  planned  systems  with  PUES  communications  ports  using  the 
Standard  Formats  for  Weather  Data  Exchange  Among  Automated  Weather  Information 
Systems.  FCM-S2-1986  (Redbook)  data  formats.  In  addition,  the  specialists  identified 
several  other  concerns  with  the  use  of  PUES  ports  based  upon  a  review  of  these 
documents.  First,  there  would  be  limited  flexibility  in  the  frequency  and  type  of  products 
available  over  the  PUP  PUES  port.  Second,  decoding  of  RPG-formatted  products  and 
reformatting  them  into  Redbook  format  would  likely  require  significant  computer  memory 
resources.  Third,  some  Redbook-formatted  products  were  estimated  to  require  more  than 
2  minutes  to  transmit.  This  would  limit  the  number  of  products  that  can  be  transmitted 
across  a  PUES  port  for  each  volume  scan. 

3.1 1 .2.2  The  test  team  specialists  also  found  other  documentation  deficiencies  that  applied 
to  all  of  the  communication  ports.  First,  information  was  not  logically  organized  in  the 
Communication  Interface  User's  Guide  (CIUG)  and  interface  control  documents  (ICDs). 
The  same  topics  were  scattered  over  several  different  documents,  but  none  of  the 
documents  contained  sufficient  information  to  stand  alone.  Second,  the  ICDs  did  not 
clearly  describe  in  detail  the  standard  communication  protocol  implementation.  A 
description  of  the  NEXRAD  products  available  for  each  interface,  and  the  format  used  to 
transmit  them,  was  missing.  The  ICDs  had  inadequate  detail  and  description  of  the 
transport,  message  format,  and  data  link  layers  of  the  communication  protocol.  In  several 
places,  information  concerning  the  physical  layer  was  marked  "To  be  determined."  Third, 
deviations  from  accepted  standards  and  protocols  were  not  clearly  noted  or  explained. 
The  documentation  did  not  explain  why  the  Advanced  Data  Communication  Control 
Procedures  standard  "flag"  definition  had  a  different  value  for  the  wideband  interface  than 
for  the  narrowband  interfaces.  Fourth,  there  were  various  inconsistencies  in  the  CIUG  and 
the  ICDs.  The  Unit/PUES  ICD  gave  default  timing  values  while  the  Unit/Principal  and  Other 
Users  ICD  did  not.  Formatted  commands  and  responses  did  not  match  in  some  of  the 
ICDs.  Also,  there  was  a  conflict  with  block  formats  and  field  identifiers  in  the  CIUG.  A 
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location  was  specified  for  a  message  code  value,  but  in  several  block  formats  there  was 
something  else  specified  for  that  location.  Although  there  appeared  to  be  sufficient 
information  to  interface  systems  with  the  "Other  Users"  ports  on  NEXRAD,  the  deficiencies 
noted  above  made  it  difficult  and  time-consuming  to  find  and  organize  this  information. 

3.1 1.2.3  For  interoperability  with  existing  systems,  DOC  operators  were  able  to  effectively 
use  the  data  from  the  hard  copy  device  to  support  their  weather  warning  verification 
process. 

3.11.3  Recommendations. 

3.11.3.1  For  JSPO: 

a.  Provide  a  stand-alone  interface  document  for  each  NEXRAD  interface.  (SRs  407, 
526,  302) 

b.  Clearly  document  deviations  from  accepted  standards  and  protocols.  (SRs  261, 

466) 

3.11.3.2  For  users:  Investigate  if  the  identified  concerns  associated  with  the  PUES  port 
will  adversely  impact  its  intended  use. 

3.12  OBJECTIVE  S-12.  Assess  NEXRAD  reliability. 

3.12.1  Method.  The  measure  of  NEXRAD  reliability  was  MTBM  (total  corrective).  The 
test  team  calculated  MTBM  (total  corrective)  as  well  as  MTBM  (inherent),  MTBM  (induced), 
and  MTBM  (no  defect)  for  both  the  system  and  the  individual  functional  areas.  The 
following  definitions  were  used  for  these  calculations: 

a.  Malfunction.  An  overall  category  of  problems  requiring  a  maintenance  response. 
Failures  and  critical  failures  (both  are  defined  in  objective  S-14)  and  LRU  malfunctions  are 
subsets  of  this  category. 

b.  Inherent  Malfunctions.  A  malfunction  resulting  from  internal  design  and 
manufacturing  characteristics. 

c.  Induced  Malfunctions.  A  malfunction  resulting  from  other  than  internal  design  and 
manufacturing  characteristics.  For  example,  improper  maintenance,  operator  error,  or 
failures  due  to  malfunction  of  associated  equipment. 

d.  No-Defect  Maintenance  Event.  A  maintenance  event  which  has  no  confirmed 
malfunction. 

3.12.1.1  The  test  team  performed  24-hour-a-day  organizational-level  maintenance  and 
collected  reliability  data  during  both  Part  A  and  Part  B  of  IOT&E(2).  However,  only  data 
collected  during  Part  B  and  reviewed  and  categorized  by  the  Data  Reduction  and  Analysis 
Working  Group  (DRAWG)  were  used  for  MTBM  calculations.  Operational  times  and 
maintenance  data  were  collected  using  MDC  forms  and  operations  logs.  These  data,  along 
with  DRAWG  categorizations,  were  entered  into  the  Micro-Omnivore  logistics  data  base. 
The  DRAWG  reviewed  all  maintenance  data  for  failures  that  required  a  maintenance 
response  and  assessed  whether  the  failures  experienced  were  inherent,  induced,  or  no¬ 
defect  maintenance  events. 
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3.12.2  Results  and  Conclusions.  The  demonstrated  MTBM  (total  corrective)  for  the 
NEXRAD  system  was  25.3  hours.  The  MTBM  (total  corrective)  and  the  number  of  inherent, 
induced,  and  no-defect  maintenance  events  for  the  system  and  each  functional  area  are 
given  in  table  111-4. 

Table  1 1 1-4 
Reliability  Data 

Category  RDA  RPG  PUP  System 


Maintenance  Events 


Inherent 

26 

20 

27 

73 

Induced 

1 

0 

7 

8 

No- Defect 

2 

3 

1 

6 

Total 

29 

23 

35 

87 

MTBM  (total  corrective) 

53.1 

78.6 

125.6 

25.3 

(hours) 


Notes:  (a)  Categorizations  by  functional  area  and  maintenance  event  types  determined 
by  DRAWG. 

(b)  PUP  maintenance  events  were  collected  from  the  three  operational  PUPs  at 
the  WSFO,  Tinker  AFB  BWS.  and  FAA  Academy.  The  PUP  data  were 
averaged  for  MTBM  determinations. 


3.12.2.1  The  demonstrated  MTBM  (inherent)  for  the  NEXRAD  system  was  29.3  hours. 
The  MTBM  (induced)  and  MTBM  (no-defect)  were  not  computed  because  of  the  limited 
number  of  induced  malfunctions  and  no-defect  maintenance  events  which  occurred  in  each 
functional  area. 

3.12.2.2  Four  reliability  problem  areas  were  identified.  The  preproduction  transmitter 
required  19  maintenance  events  in  Part  A  and  12  in  Part  B.  The  RPG  required  1 
maintenance  event  in  Part  A  and  1 2  events  in  Part  B  to  restore  operations  following  power 
transitions.  The  three  graphics  processors  required  1 2  maintenance  events  in  Part  A  and 
10  in  Part  B  to  correct  graphic  lockups.  The  four  archive  optical  disk  drive  units  required 
8  maintenance  events  in  Part  A  and  5  in  Part  B,  primarily  to  either  remove  a  jammed  disk 
or  remove  and  replace  the  entire  disk  drive  unit.  For  detailed,  additional  reliability  data 
see  appendix  D. 

3.12.2.3  Impacts  on  Maintenance  Workload: 

a.  The  agencies’  primary  weather  radars  (the  WSR-57  and  FPS-77)  have  a 
demonstrated  reliability  approximately  10  times  greater  than  the  demonstrated  reliability 
of  the  NEXRAD  tested  and  a  demonstrated  mean  time  to  repair  (MTTR)  (for  73  inherent 
malfunctions)  similar  to  NEXRAD's.  For  the  WSR-57  the  mean  time  between  failure 
(MTBF)  was  14  days  (based  on  a  1-year  average,  October  1985  through  September  1986): 
NEXRAD  was  1 .2  days.  For  the  FPS-77  the  mean  time  between  critical  failure  (MTBCF) 
was  18.5  days  (based  on  a  2-year  average,  July  1987  through  June  1989);  NEXRAD  was 
1.9  days.  The  MTTR  was  2.5  hours,  4.4  hours,  and  3.5  hours  for  the  WSR-57,  FPS-77, 
and  NEXRAD,  respectively.  The  decreased  reliability  and  the  similar  maintainability 
indicated  that  NEXRAD  will  increase  the  workload  for  technicians  at  maintenance  locations 


responsible  for  an  entire  NEXRAD  system.  In  many  cases,  one  or  more  trips  per 
maintenance  event,  to  a  remotely  located  RDA  (up  to  35  miles  away),  will  be  required  to 
restore  system  operations.  If  both  the  RDA  and  RPG  are  remotely  located,  the  system 
will  have  an  even  greater  impact  on  maintenance  workload  because  multiple  trips  may  be 
required  to  obtain  additional  spares,  materials,  etc. 

b.  For  PUP  only  sites,  NEXRAD’s  demonstrated  reliability  and  repair  times  in 
comparison  with  those  of  the  agencies’  primary  radar  showed  that  the  NEXRAD  system 
may  have  little  or  no  impact  on  maintenance  workload.  However,  maintainors  still 
expressed  concern  about  the  repeated  maintenance  responses  for  graphics  processor 
problems,  primarily  corrected  through  only  a  reseating  of  the  processor  hard  cursor  card. 

3.12.2.4  Operator  Reset/Restart  Actions.  The  operators  were  required  to  perform  5 
software  resets/restarts  on  the  RDA.  22  on  the  RPG,  and  400  on  the  three  operational 
PUPs.  These  operator  actions  occurred  in  Part  A  and  Part  B  and  were  not  included  in 
reliability  and  maintainability  calculations  because  no  maintenance  actions  (as  categorized 
by  the  DRAWG)  were  required.  The  majority  of  the  operator  resets/restarts  at  the  PUPs 
were  required  to  correct  graphics  processor  lockups. 

3.12.3  Recommendations.  For  JSPO: 

a.  Assess  transmitter  reliability  and  take  appropriate  action  to  correct  recurring 
transmitter  problems.  (SRs  098B,  002B,  1 1 2A,  1 49) 

b.  Correct  the  problems  associated  with  the  RPG  failing  to  recover  automatically 
after  power  transfers.  (SRs  317.  087B) 

c.  Determine  the  underlying  causes  of  Ramtek  graphics  problems  and  take 
appropriate  action  to  eliminate  recurrence.  (SRs  083,  301,  418) 

d.  Correct  the  problems  associated  with  the  optical  disk  drive  unit  and  archive 
functionality.  (SRs  368.  061,  332,  051) 

e.  For  all  other  failures,  determine  the  failure  sources  and  take  corrective  action. 
3.13  OBJECTIVE  S-13.  Evaluate  NEXRAD  maintainability. 

3.13.1  Method.  During  IOT&E(2)  the  test  team  performed  organizational- level  maintenance 
using  the  PTM  and  provided  support  equipment.  The  test  team  recorded  data  on  all 
failures,  to  include  associated  maintenance  times  for  troubleshooting  (isolation),  repair,  and 
verification  of  corrective  action.  However,  only  data  collected  during  Part  B,  reviewed 
and  categorized  by  the  DRAWG,  was  used  for  the  maintainability  calculations.  The  test 
team  collected  maintainability  data  using  MDC  forms  and  operations/maintenance  logs. 
These  data,  along  with  the  DRAWG  categorizations,  were  entered  into  the  Micro-Omnivore 
logistics  data  base.  Based  on  the  DRAWG  categorizations,  the  test  team  calculated  the 
MTTR  for  LRU  malfunctions,  hardware  failures,  and  all  inherent  malfunctions,  the 
percentage  of  LRU  malfunctions  isolated  using  primary  fault  isolation  (PFI),  and  the  mean 
time  to  troubleshoot  (MTT). 

3.13.1.1  The  test  team  collected  data  on  the  percentage  of  failures  that  were  identified 
by  on-line  fault  monitoring,  along  with  information  on  cannot  duplicate  (CND)  events  and 
false  alarms,  to  assess  the  adequacy  of  system  status  monitoring.  The  test  team  also 
compiled  data  to  assess  the  adequacy  of  logistic  support  elements,  including  training, 
technical  data,  diagnostics,  support  equipment,  spares,  and  facilities,  as  it  applied  to  both 
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scheduled  and  unscheduled  maintenance  actions.  This  was  accomplished  primarily  by  test 
team  observations  and  maintainability  questionnaires. 

3.13.1.2  The  primary  measure  of  maintainability  was  MTTR  (for  LRU  malfunctions).  The 
primary  measures  of  NEXRAD  fault  isolation  performance  were  the  percentage  of  LRU 
malfunctions  isolated  to  one  LRU  using  PFI  and  the  percentage  of  LRU  malfunctions 
isolated  to  three  or  fewer  LRUs  using  PFI. 

3.13.2  Results  and  Conclusions.  NEXRAD  did  not  meet  the  users’  requirements  for 
MTTR,  PFI  isolation  to  a  single  LRU,  or  PFI  isolation  to  a  group  of  three  or  fewer  LRUs 
(see  table  111-5). 


Table  1 1 1-5 


LRU  Maintainability 
Users’ 

Requirements 

MTTR  0.5  hours 

PFI  to  single  LRU  80% 

PFI  to  3  or  fewer  LRUs  95% 


Results 


9.0  hours 
50% 
57% 


NOTE:  The  MTTR  was  computed  based  upon  the  1 6  LRU  malfunctions  that  occurred 
during  IOT&E(2)  Part  B.  Two  of  the  16  were  not  used  in  computing  the  PFI  values 
above,  because  the  use  of  PFI  was  not  involved. 


3.1C.2.1  For  the  29  hardware  failures  during  Part  B  (including  the  16  LRU  malfunctions 
used  above),  the  functional  area  and  system  MTTR  and  MTT  are  given  in  table  MI-6.  The 
test  team  could  not  repair  3  of  the  29  system  hardware  failures  (including  2  of  the  LRU 
malfunctions)  and  had  to  request  field  maintenance  services  in  accordance  with  the 
Contractor  Support  Services  Plan  (CSSP). 


Table  111-6 

Hardware  Maintainability 


RDA 

RPG 

PUP 

System 

MTTR  (hours) 

18.0 

0.2 

2.0 

6.9 

MTT  (hours) 

16.0 

0.0* 

1.4 

6.0 

*Less  than  1  minute  of  troubleshooting  time  was  required  for  the  1  RPG  hardware 
failure. 


3.13.2.2  For  the  73  inherent  malfunctions  during  Part  B  (including  the  29  hardware  failures 
used  above),  the  functional  area  and  system  MTTR  and  MTT  are  given  in  table  111-7. 
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Table  111-7 


Inherent  Failures  Maintainability 


RDA  RPG  PUP  System 


MTTR  (hours)  7.1 

MTT  (hours)  6.1 


1.0  1.9 

0.7  1.5 


3.5 

2.9 


3.13.2.3  The  three  primary  deficiencies  that  contributed  to  the  system  MTTR  were  training 
(see  objective  ES-9),  the  PTM  (see  objective  S-15),  and  PFI. 

3.1 3.2.4  PFI  contributed  to  the  fault  isolation  process  in  73.9  percent  of  the  troubleshooting 
actions.  When  PFI  did  contribute,  the  MTT  was  3.1  hours.  In  those  cases  where  PFI  did 
not  contribute,  the  MTT  was  1 9.2  hours.  The  concept  of  PFI  for  the  N EXRAD  system  can 
be  divided  into  three  areas:  (1)  PTM  fault  isolation  flowcharts,  (2)  on-line  diagnostics, 
and  (3)  off-line  diagnostics.  During  IOT&E(2),  all  were  inadequate  for  isolating  faults  in 
the  N EXRAD  system  within  the  maintenance  concept. 

a.  The  PTM  fault  isolation  flowcharts  had  limited  usefulness  as  the  primary  fault 
isolation  tool.  The  flowcharts  were  incomplete  and  ambiguous  and  they  contained 
numerous  errors.  For  the  majority  of  maintenance  events,  the  flowcharts  led  technicians 
to  the  incorrect  area  or  failed  to  isolate  the  fault.  In  many  cases,  the  flowcharts  indicated 
failed  LRU(s)  that,  when  replaced,  did  not  correct  the  problem.  The  maintainors’ 
assessment  indicated  that  the  flowcharts  must  be  supplemented  by  quality  training  and 
comprehensive  documentation.  In  addition,  technical  procedures  (secondary  fault  isolation) 
to  augment  and  back  up  the  PFI  (to  allow  maintenance  personnel  to  isolate  faulty  LRUs 
using  standard  support  equipment)  did  not  exist. 

b.  The  on-line  diagnostics  use  of  built-in  test  (BIT)  and  self-diagnostic  logic  seemed 
to  be  sufficiently  integrated  within  the  system:  however,  several  identified  problems  limited 
its  benefit.  First,  the  documentation  failed  to  provide  adequate  information  on  error 
codes/rr.essages,  and  a  thorough  description  of  self-diagnostic  tests  was  not  provided. 
Second,  BIT  was  not  sufficient  to  isolate  malfunctions  to  a  specific  LRU.  Finally,  the 
number  of  system  status  messages/alarms,  many  of  which  were  false  alarms,  negated 
their  usefulness  as  a  fault  isolation  tool. 

c.  Off-line  diagnostics  were  not  sufficient  to  isolate  faults.  The  Radar  Data 
Acquisition  System  Operational  Test  (RDASOT)  had  several  baseline  failure  indications. 
Also,  when  using  RDASOT,  failure  indications  often  did  not  indicate  further  maintenance 
actions  and  faults  could  not  be  isolated  through  further  use  of  the  flowcharts.  Adequately 
detailed  documentation  for  each  off-line  diagnostic  was  not  available. 

3.13.2.5  The  system  MTTR  was  greatly  impacted  by  the  average  length  of  the  RDA 
restoration  times.  The  demonstrated  RDA  MTTR  of  18.0  hours  was  attributable  to  several 
key  maintainability  issues.  Other  than  the  diagnostics  problems  noted  above  and  the  PTM 
problems  (see  objective  S-15),  the  most  significant  problem  was  that  technicians  were  not 
provided  thorough  training  on  RDA  functionality,  theory  of  operation,  the  use  of  complex 
test  equipment,  and  key  software  and  hardware  interrelationships.  In  three  of  the  RDA 
failures,  the  test  team  could  not  repair  the  system  and  requested  field  maintenance  services 
in  accordance  with  the  CSSP.  In  these  cases  the  MTTR  of  49.9  hours  included  both  test 
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team  and  contractor  maintenance  actions.  When  the  test  team  was  able  to  repair  RDA 
failures,  the  MTTR  was  1.6  hours. 

3.13.2.6  The  test  team  completed  54  PMIs  during  IOT&E(2)  Part  B.  The  actual  time 
required  to  complete  these  PMIs  totaled  8.1  hours.  The  time  requirements  listed  in  the 
PTM  for  these  PMIs  totaled  7.3  hours.  The  test  team  estimated  downtime  for  PMIs  will 
be  33.2  hours  per  year  compared  to  24  hours  required  by  the  maintenance  concept.  This 
was  based  on  the  30.4  hours  per  year  specified  in  table  5-3.1  of  the  PTM  and  the  2.8 
hours  per  year  for  PMIs  which  will  require  downtime  but  were  not  specified  as  such  in  the 
PTM  (i.e.,  operational  check  of  Micro  Junior  control  panel,  the  transmitter  pulse  width 
check,  and  the  Klystron  spectrum  check). 

3.13.2.7  The  on-line  system  status  monitoring  system  generated  status  aiarms/messages 
so  frequently  that  PUP  and  UCP  operators  often  ignored  them,  even  though  some  indicated 
"maintenance  mandatory."  Under  minimal  load  conditions,  with  four  dedicated  PUPs 
connected  to  the  system,  approximately  45  system  status  alarms/messages  were  generated 
per  hour  and  displayed  at  the  UCP.  Many  of  these  alarms/messages  reflected 
communications  connects/disconnects,  narrowband  overload/loadshedding,  and  transmitter 
peak  power  low.  Under  a  representative  test  load  of  1 9  users,  system  status 
alarms/messages  in  excess  of  90  per  hour  were  noted. 

a.  Because  of  the  number  of  system  status  messages,  the  test  team  was  unable 
to  investigate  all  aiarms/messages  to  determine  which  were  unconfirmed  fault  indications 
(UFIs).  The  test  team  found  that  sometimes  the  system  operated  without  generating  any 
UFIs;  however,  the  operator  usually  received  at  least  one  UFI  per  hour  at  the  UCP.  When 
the  RDA  was  unstable,  operators  noted  as  many  as  seven  UFIs  in  1  hour.  Most  of  these 
indicated  a  degradation  in  the  transmitter/receiver  circuitry;  however,  many  indicated 
hardware  failures.  As  a  result  of  inadequate  training,  documentation,  and  the  large  number 
of  system  status  alarms/messages,  the  operators  often  did  not  know  what  actions  to  take 
or  what  effect  the  alarms/messages  may  have  had  operationally. 

b.  On-line  system  status  monitoring  identified  69  percent  of  the  failures  that  required 
maintenance  actions  during  IOT&E(2)  Part  B.  However,  as  noted  above,  operators  often 
ignored  the  alarms  until  they  noted  system  degradation.  The  system  status  monitoring 
normally  did  not  help  in  the  cases  of  Ramtek  graphics  processor  lockups  because  the 
operator  would  realize  the  graphics  were  inoperable  usually  about  20  seconds  before  the 
system  indicated  a  problem  existed. 

c.  The  percentage  of  organizational  CND  maintenance  events  during  IOT&E(2)  Part 
B  was  7.9  percent.  The  mean  time  spent  troubleshooting  CNDs  was  0.5  hour.  This 
decreased  from  the  values  determined  for  Part  A  (10.5  percent  and  1.0  hour,  respectively). 
Of  the  six  CND  events  experienced  during  Part  B.  two  involved  RDA  alarms  which  cleared 
without  maintenance  intervention.  Two  other  times  the  operator  received  archive  error 
messages,  but  the  archive  was  operational  before  maintenance  technicians  arrived.  The 
last  two  were  RPG  communication  alarms;  one  the  technicians  were  unable  to  duplicate, 
the  other  the  system  recovered  automatically  before  maintenance  responded. 

3.13.2.8  Another  key  maintainability  issue  was  that  maintenance  technicians  could  not 
verify  system  calibration  or  accuracy.  The  RDA  calibration  described  in  the  PTM  was 
complex,  erroneous,  and  confusing.  It  primarily  consisted  of  checking  test  signal  path 
losses  and  did  not  provide  an  end-to-end  RDA  calibration.  Maintenance  technicians  could 
not  verify  what  effect  changing  the  calibration  parameters  had  on  the  system  and  could 
not  verify  that  the  system  was  correctly  monitoring  transmitter  output  power.  The  RDA 
calibration  file,  to  which  the  technicians  had  access,  contained  more  than  200  adaptable 
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parameters.  However,  the  documentation  did  not  show  the  nominal  range  of  parameter 
values,  how  each  parameter  should  be  used,  when  or  why  it  should  be  changed,  or  how 
changing  the  parameter  would  affect  system  calibration.  After  receiver  alignments  were 
completed,  maintenance  technicians  were  unable  to  verify  if  correct  reflectivity  and  velocity 
values  were  displayed  by  the  system.  During  the  Pedestal  Alignment  Check  (suncheck), 
used  to  verify  system  positional  accuracy,  the  system  would  not  accept  the  updated 
correction  factors  and  the  technicians  were  not  provided  enough  information  to  complete 
the  check. 

3.13.2.9  The  test  team  documented,  in  service  reports,  deficiencies  that  impacted  system 
accessibility  and  ease-of -maintenance.  Besides  technical  data  deficiencies,  the  two  most 
common  problems  noted  were  missing  or  incorrect  labeling/reference  designators  and  the 
poor  design  of  cable  routing/terminations  within  the  equipment  cabinets.  However,  two  of 
the  most  significant  ease-of-maintenance  problems  noted  were  the  lack  of  storage  space 
in  the  generator  shelter,  and  the  lack  of  storage  and  work  space  for  maintenance  activities 
in  the  RDA  shelter. 

3.13.2.10  Skill  levels  of  agency  technicians  that  participated  in  I0T&E(2)  ranged  from  a 
5-level  technician  with  4  years  of  experience  to  a  journeyman  with  36  years  of  experience. 
With  the  training,  technical  manuals,  and  diagnostics  provided  for  IOT&E(2).  the  test  team 
agency  technicians  were  not  able  to  maintain  the  system  within  the  required  time  to  repair. 

3.13.3  Recommendations.  For  JSPO: 

a.  Ensure  the  technical  data  are  adequate  to  maintain  the  system.  (SRs  014,  265A. 
129,  440,  138A,  227,  328,  384,  138,  286,  420,  544,  251,  543,  403,  239,  169A,  285,  416, 
439,  077) 

b.  Resolve  all  training  issues  impacting  maintainability  (see  objective  ES-9). 

c.  Ensure  on-line  fault  monitoring  is  improved  by  reducing  the  frequency  of  system 
status  alarms/messages  and  by  eliminating  unconfirmed  fault  indications.  (SRs  531.  072B. 
006B,  129.  530,  168A,  437,  138A,  009A,  139A,  104A,  439) 

d.  Ensure  the  system  on-line  BIT  and  self-diagnostics  are  improved  to  consistently 
and  accurately  isolate  faults  within  specific  areas/subsystems.  (SRs  072B.  439.  21 2.  248. 
300,  057,  541,  255A,  386,  467,  213) 

e.  Ensure  all  off-line  diagnostic  tests  are  improved  so  that  LRU  malfunctions  can 
be  isolated  within  the  criteria  specified  by  the  maintenance  concept.  (SRs  251,  169 A. 
264,  018,  008,  378,  170,  048,  094,  064.  066,  117,  245,  151,  082.  141.  368.  065.  471. 
319,  470,  469) 

f.  Provide  sufficient  storage  and  workspace  for  maintenance  in  the  RDA  and 
generator  shelters.  (SRs  187,  047B,  096) 

g.  Ensure  all  equipment/LRUs  are  correctly  labeled  and  cable  routing  and  terminations 
are  designed  for  ease-of-maintenance.  (SRs  098,  159,  055,  208A,  124,  125A.  268.  036. 
132,  262,  312,  314,  038,  034B,  292.  347,  144,  472,  134,  232,  070B,  148,  099,  045.  030, 
156,  145,  105,  054,  116,  171,  114) 

h.  Provide  secondary  fault  isolation  procedures  to  augment  and  back  up  the  PFI. 
(SR  169  A) 
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3.14  OBJECTIVE  S-14.  Evaluate  N EXRAD  availability. 


3.14.1  Method.  The  NEXRAD  system  availability  was  measured  in  terms  of  full  system 
availability  and  degraded  system  availability. 

3.14.1.1  The  test  team  collected  availability  data  during  both  Part  A  and  Part  B;  however, 
only  data  collected  during  Part  B  and  reviewed  and  categorized  by  the  DRAWG  were 
used  for  availability  calculations.  The  test  team  collected  data  on  operational  hours, 
failures,  maintenance  actions,  and  downtimes  on  MDC  forms  and  operations  logs.  These 
data,  along  with  the  DRAWG  categorizations,  were  entered  into  the  Micro-Omnivore 
logistics  data  base.  As  appropriate,  the  DRAWG  determined  whether  each  failure  impacted 
availability  and  whether  it  was  a  critical  or  noncritical  failure. 

3.14.1.2  The  test  team  calculated  full  system  and  degraded  system  operational  availability 
(A0)  for  the  system  as  well  as  for  each  functional  area.  Based  on  failures  requiring  a 
maintenance  response.  A0  was  computed  using  inherent  failures,  as  well  as  those  no¬ 
defect  and  induced  failures  attributable  to  equipment  design.  The  following  definitions  and 
methods  were  used  when  collecting  and  analyzing  operational  availability  data: 

a.  Full  system  operational  availability  was  based  on  the  capability  to  perform  the 
functions,  except  Archive  I  and  II,  shown  in  the  NEXRAD  Unit  Operational  Functional  Flow 
Diagram  (figure  3.4  of  the  1984  NEXRAD  Technical  Requirements  (NTR)  included  here 
as  figure  111-1).  Thus,  any  failure  which  prevented  the  system  from  performing  any  of  the 
functions  in  figure  111-1  (except  Archive  I  and  II)  impacted  full  system  A0. 

b.  Degraded  system  operational  availability  was  based  on  the  capability  to  perform 
the  following  critical  functions  outlined  in  the  NTR,  figure  3.4  and  table  3.6  (key  operational 
functions  and  subfunctions,  respectively):  1  (transmit/receive),  2  (signal  processing- 
reflectivity),  4  (base  product  generation/distribution),  and  8a  (display  locally  stored  base 
products).  Thus,  any  critical  failure  which  prevented  the  system  from  performing  any  of 
these  four  critical  functions  impacted  the  degraded  system  A0. 

3.14.2  Results  and  Conclusions. 

3.14.2.1  The  NEXRAD  full  system  operational  availability  of  86.3  percent  did  not  meet 
the  users’  requirement  of  90  percr-nt. 

3.14.2.2  The  NEXRAD-degraded  system  operational  availability  of  88.2  percent  did  not 
meet  the  users’  requirement  of  96  percent. 

3.14.2.3  During  IOT&E(2)  Part  B,  the  system  experienced  70  inherent  failures  (hardware 
and  software)  which  impacted  availability;  47  were  critical.  The  DRAWG  categorized  all 
failures  based  on  failure  definitions  in  the  NTR. 

a.  Since  the  integrated  logistics  support  was  not  available  at  the  beginning  of 
IOT&E(2),  the  system  availability  calculations  were  based  on  the  assumptions  in  appendix 
C  of  the  NEXRAD  Maintenance  Concept.  These  assumptions  include  a  95-percent  sparing 
level  (spares  available  on-site  to  repair  95  percent  of  the  LRU  failures)  and  a  24-hour 
response  time  for  the  remaining  5  percent  of  the  LRU  failures.  This  resulted  in  a  constant 
administrative  and  logistics  delay  time  of  2.2  hours  (for  LRU  replacement)  or  1.0  hours 
(for  non-LRU  replacement)  being  added  to  each  maintenance  action  as  validated  by  the 
DRAWG. 
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Figure  111-1.  NEXRAO  Unit  Operational  Functional  Flow  Diagram 


b.  In  accordance  with  the  definitions  in  the  test  plan,  the  downtime  calculations 
included  the  repair  time  for  all  failures  (not  just  the  LRU  failures)  during  Part  B.  The 
operational  availability  data,  as  well  as  the  reliability  and  maintainability  data  used  to 
compute  the  availability,  for  the  system  and  functional  areas  are  given  in  table  III-8. 


MTBF 

(hours) 

M 

(hours) 

Table  111-8 

RM&A  Data 

Ao(full) 

(%) 

MTBCF 

(hours) 

Mcf 

(hours) 

Area 

A^degraded) 

(%) 

System 

30.7 

4.9 

86.3 

44.8 

6.0 

88.2 

RDA 

64.2 

9.0 

87.7 

96.3 

11.7 

89.2 

RPG 

90.4 

2.0 

97.8 

120.5 

2.2 

98.2 

PUP 

Where: 

169.0 

3.2 

98.1 

274.6 

3.9 

98.6 

MTBF  is  the  mean  time  between  failure 

MTBCF  is  the  mean  time  between  critical  failure 

M  is  the  mean  downtime 

Mcf  is  the  mean  downtime  for  critical  failures 

A0(full)  is  the  full  system  operational  availability 

AQ(degraded)  is  the  degraded  system  operational  availability 

NOTE:  After  the  computations  were  performed,  the  results  were  rounded  to  a  single 
decimal  point. 


3,14.2.4  As  shown  above,  the  RDA  availability  had  the  greatest  impact  on  system 
availability.  The  largest  detractor  from  the  RDA  availability  was  the  preproduction 
transmitter  reliability  and  maintainability  problem  (11  transmitter  failures  with  a  10.4-hour 
MTTR).  Although  the  RPG  failures  associated  with  power  transitions  did  not  have  a 
significant  impact  on  overall  system  availability,  the  timing  of  these  events  critically 
degraded  the  effectiveness  of  NEXRAD  as  an  aid  in  providing  required  weather  support. 
Many  times  in  severe  weather  episodes,  when  the  operators  needed  the  system  to  support 
critical  weather  warning  operations,  the  system  was  not  available.  When  the  operators 
considered  the  overall  system  performance,  including  the  impact  of  availability,  for 
supporting  the  mission  requirements,  they  stated  that  the  system  did  not  meet  their 
minimum  operational  requirements. 

3.14.3  Recommendations. 

3.14.3.1  For  JSPO: 

a.  Ensure  RDA  reliability  and  maintainability  problems,  particularly  with  the  transmitter, 
are  resolved.  (SRs  098B,  206,  002B,  1 12A,  420,  207,  354,  096,  026,  149,  036,  078.  033B. 
267,  483,  353,  185,  327,  073,  095,  118) 

b.  Ensure  RPG  power  transfer  problems  are  resolved.  (SRs  317,  087B,  166) 

c.  Ensure  overall  system  reliability  and  maintainability  problems  are  re  solved  (see 
objectives  S-12  and  S-13). 
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3.14.3.2  For  JSPO  and  users:  Ensure  sparing  level  is  adequate  to  meet  availability 
requirements. 

3.15  OBJECTIVE  S-15.  Assess  the  adequacy  of  logistics  support. 

3.15.1  Method.  The  provisioning  process  for  support  equipment  and  spares  will  not  be 
completed  until  mid-1990.  Therefore,  a  limited,  contractor-proposed,  JSPO-approved 
package  of  support  equipment  and  spares  was  used  for  10T&E(2).  Also,  since  the 
production  technical  orders  are  not  deliverable  until  1990,  the  PTM  was  used  during  the 
test.  Deficiencies  in  the  PTM  that  adversely  affected  the  performance  of  maintenance  were 
documented.  The  test  team  also  documented  other  discrepancies  in  the  technical  manuals 
that  did  not  affect  IOT&E{2)  maintenance  actions.  The  test  team  assessed  the  sufficiency 
of  support  equipment  by  using  support  equipment  as  prescribed  in  the  PTM  for 
maintenance  activities.  The  test  team  assessed  the  adequacy  of  on-site  spares  by 
collecting  hardware  failure  and  parts  consumption  data.  Agency  provisioning,  equipment, 
and  logistics  specialists  reviewed  logistics  support  planning  documents  to  address  the 
adequacy  of  planned  provisioning  to  support  their  agencies’  requirements. 

3.15.2  Results  and  Conclusions. 

3.15.2.1  The  PTM  and  the  JSPO  list  of  required  support  equipment  din  not  agree.  The 
test  team  identified  20  items  of  support  equipment  required  by  the  PTM  for  maintenance 
actions  which  were  not  on  the  JSPO  list.  These  deficiencies  limited  the  maintainers’ 
ability  to  perform  or  complete  scheduled  and  unscheduled  maintenance  actions. 

3.15.2.2  The  JSPO-provided  complement  of  on-site  spares  was  not  adequate  to  maintain 
the  system  in  accordance  with  the  maintenance  concept.  Therefore,  the  provisioning 
process  must  identify  a  sparing  level  that  is  better  aligned  to  the  agencies’  requirements 
than  the  contractor-proposed,  JSPO-approved  package  of  spares  that  was  provided  for 
IOT&E(2).  Of  the  56  LRU  replacements  required  during  IOT&E(2),  16  spares  (28.6 
percent)  were  on  site  and  the  remaining  40  spares  (71 .4  percent)  had  to  be  ordered.  In 
addition,  the  fault  isolation  procedures  often  required  the  technician  to  obtain  and  insert 
multiple  spare  LRUs  to  isolate  faults.  Unless  the  required  spares  were  on  site,  the  test 
team  could  not  proceed  with  fault  isolation  using  the  PTM  flowcharts  until  the  spares  were 
received.  Thus,  system  operational  effectiveness  was  severely  impacted  while  waiting  for 
spare  LRUs  not  on  site.  Also,  upon  receipt.  1 1  of  the  40  contractor-provided  spares  were 
incompatible  with  the  unit  being  tested  in  IOT&E(2).  The  majority  of  the  incompatible  LRUs 
were  for  the  RDA. 

3.15.2.3  The  unanimous  opinion  of  the  maintenance  technicians  was  that  the  PTM,  which 
includes  the  vendor  manuals,  was  inadequate  for  training  and  for  maintaining  the  NEXRAD 
system. 

a.  Although  the  contractor’s  latest  PTM  revision  (Revision  C.1)  was  an  improvement 
over  the  version  available  at  the  beginning  of  test,  it  was  still  seriously  deficient  in  many 
areas.  The  PTM  was  incomplete  and  ambiguous  and  contained  numerous  errors.  As 
a  result,  maintenance  technicians  stated  that  the  PTM  was  inadequate  for  83  out  of  159 
maintenance  actions  documented  on  maintenance  incident  questionnaires  during  IOT&E(2). 
Additionally,  of  the  72  PM  Is  scheduled  during  Part  B,  42  had  documented  technical  data 
deficiencies;  1 7  of  these  42  could  not  be  completed. 

b.  The  PTM  fault  isolation  flowcharts  had  limited  usefulness  as  the  primary  fault 
isolation  tool.  The  flowcharts  were  incomplete  and  ambiguous,  and  they  contained 
numerous  errors.  For  the  majority  of  maintenance  events,  the  flowcharts  led  technicians 


to  the  incorrect  area  or  failed  to  isolate  the  fault.  In  many  cases,  the  flowcharts  indicated 
failed  LRU(s)  that,  when  replaced,  did  not  correct  the  problem.  The  maintainors’ 
assessment  indicated  that  the  flowcharts  must  be  supplemented  by  quality  training  and 
comprehensive  documentation.  In  addition,  technical  procedures  (secondary  fault  isolation) 
to  augment  and  back  up  the  PFI  (to  allow  maintenance  personnel  to  isolate  faulty  LRUs 
using  standard  support  equipment)  did  not  exist. 

c.  The  PTM  did  not  include  procedures  for  the  organizational-level  technician  to 
verify  that  the  system  was  properly  calibrated.  The  RDA  calibration  alignment,  described 
in  the  PTM,  was  not  a  true  RDA  calibration  but  a  check  of  the  path  losses.  In  addition, 
because  of  inadequate  procedures  and  functionality,  the  test  team  could  not  complete  the 
important  Pedestal  Alignment  Check  (suncheck)  to  verify  system  positional  accuracy. 

d.  The  planned  cadre  training  will  likely  be  ineffective  if  the  technical  manuals  do 
not  have  a  major  upgrade  prior  to  its  start.  Until  the  technical  manuals  are  complete  and 
the  ambiguities  and  errors  are  removed,  the  NEXRAD  system  will  probably  not  be 
maintainable  in  accordance  with  the  NEXRAD  maintenance  concept  and  operational 
effectiveness  will  likely  be  adversely  impacted. 

3.15.3  Recommendations. 

3.15.3.1  For  JSPO  and  users: 

a.  Ensure  the  technical  aata  identify  all  support  equipment  required  to  complete 
organizational-level  maintenance.  (SRs  202,  078,  023,  483,  209,  192,  536.  107,  236) 

b.  Ensure  sparing  level  is  adequate  to  meet  availability  requirements. 

3.15.3.2  For  JSPO: 

a.  Ensure  spares  provided  are  compatible  with  the  fielded  unit  (e.g.,  limited  production 
spares  for  limited  production  equipment).  (SRs  206,  480,  099) 

b.  Ensure  the  technical  data  are  significantly  upgraded,  validated,  and  verified  well 
before  the  cadre  training  to  meet  both  the  theory  and  hands-on  training  requirements. 
(SRs  014,  265A,  129,  440,  138A,  227,  328.  384,  138,  286,  420,  544.  251) 

c.  Ensure  all  alignments  and  PMI  procedures  that  are  required  to  maintain  the 
NEXRAD  system  are  con  .ct  and  included  in  the  technical  data.  (SRs  440,  164A.  384. 
420,  169,  354,  285,  197,  355,  482,  005,  112,  412,  535,  312,  353) 

d.  Ensure  the  NEXRAD  system  technical  data  are  adequate  for  a  5-level  maintenance 
technician  to  maintain  NEXRAD  in  accordance  with  the  maintenance  concept. 

e.  Provide  secondary  fault  isolation  procedures  to  augment  and  back  up  the  PFI. 
(SR  169  A) 

3.16  OBJECTIVE  S-16.  Evaluate  NEXRAD  software  maintainability. 

3.16.1  Method.  Selected  software  documentation  was  evaluated  at  the  CPCI  level  for 
its  overall  contribution  to  the  maintainability  of  the  NEXRAD  software.  Corresponding 
NEXRAD  software  source  listings  were  evaluated  on  a  module-by-module  basis.  This 
evaluation  measured  the  extent  to  which  the  software  design,  as  reflected  in  the 
documentation  and  software  source  listings,  possessed  good  software  maintainability 
characteristics. 
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3.16.1 .1  The  AFOTEC  software  maintainability  evaluation  technique  described  in  AFOTECP 
800-2,  volume  III,  was  used  for  this  evaluation.  Ten  trained  software  evaluators  completed 
standard  questionnaires  for  the  documentation  and  selected  modules  for  four  CPCIs. 

3.16.1.2  The  evaluators  were  provided  a  software  maintainability  evaluation  guide,  which 
contained  the  questionnaires,  and  were  prebriefed  on  the  evaluation  procedures.  Although 
the  questionnaires  required  standardized  answers,  the  evaluators  included  written  comments 
as  they  deemed  appropriate. 

3.16.1.3  The  software  test  team  evaluated  the  documentation  and  source  listings  for 
CPCI-01  (RDA  Status  and  Control),  CPCI-03  (Radar  Product  Generation),  CPCI-04  (Product 
Display),  and  CPCI-28  (Performance  Monitoring  and  Data  Reduction).  Based  on  the 
response  scale  in  table  111-4  of  1  (low)  to  6  (high),  averages  of  3.5  and  above  indicate 
generally  favorable  maintainability  characteristics,  and  averages  below  3.5  indicate  generally 
unfavorable  characteristics.  Significant  deficiencies  identified  by  the  test  team  were 
reported  as  service  reports. 

3.16.2  Results  and  Conclusions.  Overall,  the  documentation  and  source  listings  met  the 
user’s  requirement  of  3.5.  The  average  scores  for  the  four  CPCIs  are  shown  in  the  tables 
111-9  and  111-10.  Each  of  the  seven  characteristics  in  the  tables  is  an  average  of  the 
questions  that  relate  to  that  characteristic.  The  overall  score  is  the  average  of  all 
questions. 


Table  111-9 


Documentation  Evaluation  Results 


Characteristic 

CPCI-01 

CPCI-03 

CPCI-04 

CPCI-28 

Modularity 

4.1 

4.3 

4.3 

3.8 

Descriptiveness 

3.3 

3.5 

3.4 

3.6 

Consistency 

3.9 

3.8 

3.8 

3.9 

Simplicity 

4.2 

4.3 

4.4 

4.7 

Expandability 

3.3 

3.5 

3.6 

3.8 

Testability 

2.9 

3.3 

3.1 

3.0 

Traceability 

3.4 

3.1 

3.3 

3.1 

Overall 

3.6 

3.7 

3.7 

3.7 

Table  111-10 


Source  Listings  Evaluation  Results 


Characteristic 

CPCI-01 

CPCI-03 

CPCI-04 

CPCI-28 

Modularity 

5.4 

5.3 

5.3 

5.2 

Descriptiveness 

4.0 

4.0 

4.1 

4.2 

Consistency 

4.0 

3.9 

3.9 

4.2 

Simplicity 

5.5 

5.3 

5.3 

5.1 

Expandability 

5.0 

5.0 

5.0 

4.8 

Testability 

4.5 

4.4 

4.5 

4.2 

Traceability 

3.3 

3.0 

3.0 

3.4 

Overall 

4.6 

4.5 

4.6 

4.6 
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3.16.2.1  Documentation.  Significant  problems  were  identified  during  the  documentation 
evaluation.  As  shown  in  the  documentation  evaluation  results  in  table  111-9,  the 
characteristics  of  testability  and  traceability  did  not  meet  the  requirement  of  3.5  for  any 
of  the  four  CPCIs.  The  characteristic  of  descriptiveness  did  not  meet  the  requirements 
for  two  of  the  four  CPCIs  evaluated.  Expandability  did  not  meet  the  requirement  for  CPCI- 
01 .  The  deficiencies  discussed  below  detail  the  primary  reasons  why  these  documentation 
characteristics  averaged  below  3.5. 

a.  The  Computer  Program  Product  Specifications  (C5)  documents  were  the  most 
significant  detractors.  The  C5s  were  inadequate  for  each  of  the  CPCIs  evaluated.  It  was 
difficult,  time-consuming,  and  sometimes  impossible  to  find  module  descriptions,  data  flow 
descriptions,  and  calling  sequences.  For  example,  the  1 ,986-page  C5  document  for  CPCI- 
01  had  a  2-page  table  of  contents  that  was  not  sufficiently  detailed,  no  index  to  find  the 
references  to  a  module,  and  no  glossary  of  unique  terms. 

b.  The  data  dictionaries  were  the  second  major  deficiency.  They  were  also 
inadequate  for  each  CPCI  evaluated.  Each  of  the  data  dictionaries  was  missing  data 
element  names,  had  no  naming  convention  to  distinguish  global  data  names  from  local 
names,  had  data  elements  from  COMMON  Blocks  that  did  not  have  the  name  of  the 
COMMON  block  listed,  made  no  distinctions  between  data  elements  being  set  or  used, 
and  had  inaccurate  data  type  information  (e.g.,  whether  global,  common,  or  local  and 
whether  a  scalar,  array,  or  literal).  For  example,  on  one  occasion,  trained  software 
evaluators  spent  approximately  3  days  unsuccessfully  trying  to  locate  information  and  trace 
the  data  flow  of  a  data  element. 

c.  A  third  deficiency  was  that  the  version  description  document  (VDD)  for  each  CPCI 
did  not  contain  adequate  descriptions.  The  files  needed  to  compile  and  link  a  CPCI  were 
not  fully  specified,  and  other  CPCIs  associated  with  or  used  by  a  CPCI  were  inadequately 
specified.  Software  evaluators  took  4  days  during  IOT&E(2)  attempting  to  build  the  CPCI- 
04  software.  They  were  unsuccessful  primarily  because  the  VDD  for  CPCI-04  did  not 
provide  adequate  compile  and  link  information. 

d.  Fourth,  it  was  the  software  evaluators’  opinion  that  adequate  test  information  was 
not  built  into  the  software  documentation.  The  evaluated  CPCI  documentation  did  not 
contain  sufficient  descriptions  of  low-level  (module)  testing  that  would  be  used  to  verify 
software  changes.  There  was  only  limited  descriptions  of  higher  level  (CPCI  and  functional 
area)  testing.  The  software  debug  tools  available  to  the  test  team  for  software  testing  were 
not  described  in  the  documentation. 

3.16.2.2  Source  Listings.  The  source  listings  were  determined  to  have  simple, 
expandable,  and  modular  characteristics.  These  characteristics  enhanced  the  maintainability 
of  the  software.  However,  the  characteristic  of  traceability  did  not  meet  the  requirements 
for  any  of  the  four  CPCIs  evaluated.  (See  table  111-10.) 

a.  An  inadequate  preface  block  in  each  module’s  source  listing  was  the  major 
deficiency  that  adversely  impacted  traceability.  Of  the  179  modules  evaluated.  165 
modules  had  errors,  inconsistencies,  or  incomplete  information.  The  data  element 
descriptions  in  the  preface  block  listed  elements  that  were  not  used  in  the  module,  did  not 
list  elements  that  were  used  in  the  module,  and  often  incorrectly  described  data  elements 
that  were  listed.  The  description  of  the  module's  function  was  either  incorrect,  inadequate, 
or  missing.  The  program  design  language  (PDL)  did  not  always  match  the  implemented 
source  code.  A  list  of  modules  which  call  the  evaluated  module  was  missing  from  the 
preface  block.  The  list  of  modules  that  the  evaluated  module  called  was  often  incorrect. 
Inadequate  data  element  descriptions  in  the  preface  blocks  caused  invalid  information  to 
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be  used  in  the  data  dictionaries.  Unless  these  deficiencies  in  the  preface  block  are 
corrected,  the  data  dictionary  program  cannot  produce  accurate  data  dictionaries. 

b.  Another  deficiency  with  the  source  listings  was  that  imbedded  comments  in  the 
source  code  were  often  just  a  repeat  of  the  PDL.  The  software  evaluators  usually  found 
no  extra  information  in  the  comments.  This  limited  the  understanding  of  the  source  code, 
especially  when  trying  to  verify  complicated  math  algorithms  in  a  module. 

3.16.2.3  The  documentation  and  source  listings  deficiencies  noted  above  severely 
degraded  the  ability  to  locate  and  trace  information  needed  to  solve  software  problems. 
Software  evaluators  stated  that  the  documentation  would  require  a  major  upgrade  before 
it  would  be  adequate  for  use  in  software  maintenance.  The  deficiencies  in  the  preface 
blocks  and  source  code  of  source  listings  identified  above  made  it  difficult  and  sometimes 
impossible  for  software  evaluators  to  trace  data  element  names,  data  flow,  and  math 
algorithm  implementations  from  the  source  code  to  the  documentation.  Although  the 
documentation  and  source  listings  met  the  users'  minimum  requirements  as  evaluated  using 
the  standard  questionnaires,  additional  personnel  and  other  resources  will  likely  be 
necessary  to  maintain  and  update  the  NEXRAD  software  unless  the  above  identified 
deficiencies  are  corrected. 

3.16.3  Recommendations.  For  JSPO: 

a.  Ensure  the  contractor  reviews  and  corrects,  for  all  modules,  the  deficiencies 
associated  with  the  preface  blocks  and  imbedded  comments  of  the  source  listings. 
(SRs  352,  496,  497,  401) 

b.  Correct  the  deficiencies  with  the  software  C5  documentation.  (SRs  335,  493,  290, 
289,  492,  374,  491,  362,  342,  498,  334,  316) 

c.  Correct  the  deficiencies  with  the  data  dictionaries.  (SR  323) 

d.  Ensure  each  version  description  document  adequately  describes  each  CPCI. 
(SRs  494,  341) 

3.17  OBJECTIVE  S-17.  Assess  the  adequacy  of  planned  and  existing  NEXRAD  software 
support  resources  (SSR). 

3.1 7.1  Method.  The  SSR  assessment  methodology  described  in  AFOTECP  800-2.  volume 
V  and  the  life  cycle  management  assessment  methodology  described  in  AFOTECP  800-2. 
volume  II  were  used  for  this  assessment. 

3.17.1.1  The  SSR  are  those  resources  required  to  accomplish  software  modifications  for 
NEXRAD.  These  resources  include  the  computers  and  associated  supporting  software, 
support  facility  layout,  personnel,  training,  test  tools,  distribution  procedures,  and 
hardware/software  documentation  required  to  accomplish,  test,  and  implement  software 
changes. 

3.17.1.2  The  test  team  assessed  whether  the  life  cycle  management  plans  addressed 
support  procedures  for  updating  and  maintaining  configuration  management.  The  draft 
Integrated  Logistics  Support  Plan  (ILSP),  Software  Management  Plan  (SMP),  and  the 
Computer  Resource  Management  Plan  (CRMP)  were  used  as  the  major  planning 
documents  in  this  assessment. 
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3.17.1.3  Ten  trained  software  evaluators  completed  the  questionnaires  for  AFOTECP 
800-2,  volumes  II  and  V.  The  Deputy  for  Software  Evaluation  (DSE)  briefed  the  evaluators 
on  the  questionnaires  and  the  assessment  procedures.  The  DSE  debriefed  the  evaluators 
after  completion  of  the  questionnaires  to  resolve  any  uncertainties  and  to  ensure  that  all 
evaluators  had  fully  addressed  each  question. 

3.17.2  Results  and  Conclusions. 

3.17.2.1  The  OSF  was  hiring  personnel  with  adequate  experience  and  skills  and  was 
working  to  obtain  more  facility  space  to  perform  various  software-related  functions  which 
were  not  yet  well  defined.  At  the  end  of  IOT&E(2),  the  NEXRAD  Computer  Resources 
Working  Group  (NCRWG)  was  still  developing  the  SMP  and  CRMP  (two  of  the  major 
project  management  plans).  OSF  management  was  taking  an  active  role  in  the 
development  of  these  documents.  However,  the  configuration  management  functions  were 
only  addressed  at  a  high  level  in  the  SMP  and  CRMP.  The  frequency  of  block  releases, 
the  procedures  of  data  handling  within  the  OSF,  and  the  assignment  of  responsibilities  were 
not  sufficiently  detailed.  A  detailed  OSF  configuration  management  plan  needs  to  be 
developed.  Since  the  project  and  configuration  management  plans  were  incomplete  (e.g., 
ILSP)  and  had  not  been  finalized  or  approved,  there  is  a  risk  that  the  OSF  resources  may 
not  be  adequate  for  the  government  to  assume  software  support  responsibilities  at  support 
management  responsibility  transfer  (SMRT). 

3.17.2.2  In  addition,  the  evaluators  identified  the  following  issues  regarding  software 
support  resources  planning.  First,  the  planned  manning  levels  of  the  OSF  appeared 
inadequate  to  monitor  the  OSF  support  contract.  Monitoring  this  contract  could  become 
a  time-consuming  and  difficult  accountability  effort  when  both  government  and  contract 
personnel  are  working  on  the  same  project.  Second,  the  personnel  and  resources  needed 
to  provide  training  for  new  employees  and  support-contract  personnel  after  the  one-time 
14-week  software  maintenance  course  were  not  addressed.  Third,  automated  support  tools 
for  software  development  and  configuration  management  were  insufficiently  addressed  in 
the  planning  documents  to  define  the  level  of  resources  required.  Automated  support  tools 
are  necessary  for  configuration  management  to  adequately  maintain  the  NEXRAD 
configuration  baseline.  Also,  without  adequate  automated  support  tools,  maintaining  and 
testing  the  N  EX  RAC  software  will  be  increasingly  difficult.  Fourth,  it  was  not  clear  if  the 
OSF,  as  currently  planned,  would  have  the  personnel  and  other  resources  necessary  to 
resolve  the  software  documentation  and  source  listings  problems  identified  in  objective 
S-16.  If  the  contractor  does  not  correct  these  deficiencies  prior  to  SMRT,  additional  OSF 
personnel  and  other  resources  will  likely  be  necessary  for  the  government  to  maintain  the 
NEXRAD  software. 

3.17.3  Recommendations. 

3.17.3.1  For  JSPO: 

a.  Ensure  sufficient  automated  support  tools  are  available  to  support  configuration 
management,  quality  assurance,  and  software  development,  test,  and  distribution.  (SR  487) 

b.  Develop  an  adequate  configuration  management  plan  for  the  OSF. 

3.17.3.2  For  JSPO  and  users:  Ensure  the  ILSP,  SMP,  and  CRMP  are  coordinated  and 
approved. 
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3.17.3.3  For  OSF: 


a.  Ensure  sufficient  resources  are  available  to  monitor  the  OSF  software  support 
contractor. 

b.  Develop  an  OJT  and  formal  follow-on  training  program  for  training  new-hires  and 
support-contract  personnel  after  the  one-time  14-week  contractor-provided  training  course. 

c.  Ensure  the  OSF  has  the  personnel  and  other  resources  necessary  to  maintain 
the  NEXRAD  software. 

3.18  OBJECTIVE  S-18.  Assess  NEXRAD  software  usability. 

3.18.1  Method.  The  test  team  assessed  the  usability  of  six  NEXRAD  software  man- 
machine  interfaces  through  the  use  of  the  Software  Usability  Questionnaire  (SUQ)  described 
in  AFOTECP  800-2,  volume  IV,  Software  Usability  Evaluators  Guide.  The  questionnaire 
addressed  the  six  software  usability  attributes  of  confirmability,  controllability,  workload 
suitability,  descriptiveness,  consistency,  and  simplicity. 

3.18.1 .1  Through  5  stmetured  interviews,  26  operations  personnel  independently  completed 
an  SUQ  for  the  PUP  interface  and  20  operators  completed  an  SUQ  for  the  UCP  interface. 
Similarly,  during  another  separate  stmetured  interview,  five  maintenance  personnel 
completed  an  SUQ  for  the  RDASOT  diagnostics,  the  Concurrent  Computer  maintenance 
diagnostics,  the  Ramtek  graphics  processor  maintenance  diagnostics,  and  the  RDA 
maintenance  control  console  (MCC)  interfaces.  The  personnel  were  trained  on  the  uses 
and  capabilities  of  these  interfaces  before  participating  in  this  assessment. 

3.18.1.2  An  overall  score  for  each  interface  was  obtained  by  averaging  the  responses 
from  the  questionnaire.  The  overall  average  scores  were  then  correlated  with  operator 
and  maintenance  personnel  comments.  Along  with  these  scores  and  comments  given 
during  the  questionnaire,  agency  specialist  comments  and  documented  deficiencies  were 
also  used  to  assess  the  usability  of  these  interfaces. 

3.18.2  Results  and  Conclusions.  The  averages  of  the  operators’  and  maintained'  SUQ 
responses,  by  interface,  are  presented  in  table  111-11.  Scores  of  3.5  and  above  indicate 
generally  favorable  characteristics,  and  scores  below  3.5  indicate  generally  unfavorable 
characteristics. 


Table  111-11 

Operations  and  Maintenance  SUQ  Results 


Operations 

Maintenance 

UCP 

PUP 

RDASOT 

Concurrent  Ramtek 

MCC 

4.0 

4.0 

3.3 

3.1  3.0 

3.0 

3.18.2.1  Operations.  Operators  stated  that  the  menu-driven  commands  enhanced  the 
PUP  and  UCP  usability.  Operators  did  not  have  to  memorize  commands  to  effectively  use 
the  applications  terminals.  In  addition,  many  product  manipulation  features  were  easily 
invoked  using  the  graphics  tablet  (e.g.,  rr-gnify,  filter,  and  recenter).  However,  the 
operators  identified  several  UCP  and  PU.'  interface  deficiencies.  First,  the  UCP 
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applications  terminal  was  unable  to  accept  rapid  keyboard  inputs.  Also,  the  PUP 
applications  terminal  would  not  execute  the  return  key  or  function  keys  when  the  screen 
was  being  updated.  As  operators  became  more  proficient  with  keyboard  menus  and 
commands,  these  two  problems  became  more  frustrating.  Much  time  was  wasted  by 
having  to  back  up  and  retype  the  missed  keystrokes  or  repeatedly  hit  the  return  key  until 
the  system  responded.  Second,  an  inadvertent  key  depressed  on  the  UCP  system 
console,  without  a  return  key,  eventually  led  to  an  RPG  failure  and  halted  operations. 
Third,  RCM  editing  and  nonassociated  RPG  dialup  procedures  at  the  PUP  were 
cumbersome.  Fourth,  the  PUP’s  extended  adaptation  data  were  not  sufficiently  documented 
and  required  extensive  use  of  hexadecimal  codes.  Fifth,  editing  procedures  for  the  UCP 
edit  screens  were  inconsistent  and  cumbersome.  Also,  different  editing  procedures  existed 
for  similar  PUP  and  UCP  edit  screens.  Finally,  operators  stated  they  had  difficulty  locating 
information  in  the  PUP  and  UCP  user’s  manuals  since  neither  manual  contained  an  Index. 

3.1 8.2.2  Maintenance.  Several  usability  deficiencies  were  noted  with  the  four  maintenance 
interfaces.  First,  for  similar  functions,  the  MCC  and  UCP  menu  structures  and  commands 
were  unnecessarily  different.  The  MCC  used  four  letter  commands  where  the  UCP  used 
a  one-or  two-letter  series  of  commands  separated  by  commas.  Second,  the  maintainers 
were  often  required  to  copy  important  information  by  hand  because  printers  were  not 
available  to  support  the  RDASOT,  Ramtek,  and  Concurrent  Computer  Corporation 
diagnostics  or  the  MCC  interface.  Third,  the  technical  documentation  for  the  MCC  interface 
and  the  RDASOT  and  Ramtek  diagnostics  did  not  adequately  define  the  proper  procedures 
or  explain  the  meaning  and  impact  of  error  and  status  messages.  The  Ramtek  diagnostic 
procedures  were  located  in  a  different  section  of  the  technical  manual  than  the  narrative 
for  the  procedures.  Also,  the  RDASOT  receiver  calibration  procedures  did  not  always 
explain  the  inputs  that  were  expected  from  the  maintainer. 

3.18.3  Recommendations. 

3.18.3.1  For  JSPO: 

a.  Ensure  adequate  PUP  and  UCP  user’s  manuals  are  provided  with  an  index. 
(SR  403) 

b.  Include  adequate  menu  editing  procedures  in  the  PUP  and  UCP  users'  manuals. 
(SRs  544,  543,  226,  333,  040,  1 75) 

c.  Provide  adequate  technical  documentation  for  the  Ramtek  diagnostics,  the 
RDASOT,  and  the  MCC  interface  to  include  meaning  and  impact  of  error  and  status 
messages.  (SRs  129,  384,  355.  048,  066,  117,  151,  479.  388) 

d.  Eliminate  any  use  of  hexadecimal  code  for  character  or  numeric  input.  (SRs  41 5. 

438) 


e.  Eliminate  the  inconsistencies  in  the  UCP  and  PUP  editing  screens.  (SRs  029, 
250,  309,  015,  175,  161,  039) 

f.  Provide  compatible  interfaces  for  the  MCC,  UCP,  and  PUP.  (SRs  057,  164,  143) 

g.  Enable  applications  terminals  to  accept  keyboard  entries  during  screen  updates. 
(SRs  167,  155) 

h.  Improve  RCM  editing  procedures.  (SRs  358,  333.  336,  428) 


111-39 


i.  Provide  effective  multiple  RPG  dialup  procedures  from  the  PUP.  (SRs  395,  393, 
394,  415,  515) 

3.18.3.2  For  JSPO  and  users:  Provide  a  print  capability  at  the  RDA,  RPG,  and  PUP  to 
support  the  Ramtek,  RDASOT,  and  the  Concurrent  Computer  Corporation  diagnostics  and 
the  MCC  interface.  (SRs  181  A,  082,  069,  125,  469) 

3.19  OVERALL  PERFORMANCE: 

a.  When  the  overall  performance  of  NEXRAD  was  considered,  the  median 
questionnaire  response  of  ail  the  operators  indicated  that  the  system  did  not  meet  their 
requirements  as  an  aid  for  preparing  weather  warnings,  weather  advisories,  and  routine 
weather  services  (see  page  A-2).  Most  operators  stated  that  NEXRAD  was  often  not 
available  to  support  these  services  because  of  PUP  lockups,  system  outages,  and  problems 
with  recovering  automatically  from  power  transitions.  However,  possibly  because  of  their 
smaller  area  of  weather  support  responsibilities,  DOD  median  questionnaire  responses 
indicated  that  the  system  met  their  minimum  operational  needs  when  the  overall  NEXRAD 
performance  was  considered. 

b.  When  the  operators  considered  the  overall  responsiveness  of  the  system  in  a 
multiple  user  environment,  the  median  questionnaire  response  of  the  operators  indicated 
that  the  system  met  their  minimum  operational  needs  (see  page  A-2).  However,  possibly 
because  of  their  larger  weather  support  areas,  DOC  and  DOT  median  questionnaire 
responses  indicated  that  the  system  did  not  meet  their  minimum  operational  needs  when 
the  overall  NEXRAD  responsiveness  was  considered. 

3.20  FOLLOW-ON  OPERATIONAL  TEST  AND  EVALUATION  (FOT&E).  To  refine 
estimates  of  operational  effectiveness  and  suitability,  to  evaluate  changes  and  modifications 
made  to  correct  deficiencies,  and  to  evaluate  suggested  enhancements  identified  during 
IOT&E(2),  the  using  agencies  should  address  the  following  areas  during  FOT&E. 

3.20.1  Operations: 

a.  FOT&E  should  be  performed  in  an  operational,  multiuser  environment  that  includes 
associated  and  nonassociated  PUPs  from  all  using  agencies.  To  fully  evaluate  the 
maximum  processing  load,  the  agencies  should  conduct  FOT&E  during  a  significant  weather 
season.  During  FOT&E,  NEXRAD  should  be  the  only  weather  radar  that  operators  use 
to  meet  information  dissemination  requirements. 

b.  The  system  evaluated  during  FOT&E  should  include  limited  and/or  full-scale 
production  phase  capabilities.  The  new  capabilities  that  should  be  tested  during  FOT&E 
include  the  hydrology  algorithms,  full  RPGOP  communications  speed,  and  the  limited  and 
full-scale  production  phase  algorithms. 

3.20.2  Logistics.  Organizational-level  maintenance  should  be  performed  on  a  production- 
model  NEXRAD  using  validated  and  verified  technical  manuals,  the  integrated  logistics 
support  infrastructure,  and  representative  training. 

3.20.3  Software.  To  test  government  software  support  resources  during  FOT&E,  the  OSF 
should  generate  and  test  a  new  software  version  release  to  include  adding,  deleting,  and 
changing  functionality  within  the  RDA,  RPG,  and  PUP.  This  should  be  accomplished  well 
in  advance  of  the  SMRT. 
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SECTION  IV  -  SERVICE  REPORTS 


4.0  SERVICE  REPORT  STATUS.  The  test  team  identified  deficiencies  and  enhancements. 
Service  reports  (SRs)  were  written  and  provided  to  the  JSPO  for  disposition  in  accordance 
with  Air  Force  TO  00-35D-54.  The  status  of  SRs  documented  or  revalidated  during 
IOT&E(2)  is  outlined  in  table  IV-1. 


Table  IV-1 


Status  of  Service  Reports 
(As  of  13  Aug  1989) 


Cateqory 

Identified 
Durino  IOT&E(2) 

Revalidated 
Durino  IOT&E(2) 

Total  Open 

Category  1 

9 

3 

12 

Category  II 

Deficiencies 

477 

84 

561 

Enhancements 

59 

23 

82 

Total 

545 

110 

655 

DEFINITIONS: 

a.  Category  I.  A  deficiency  that  required  immediate  corrective  action  because: 

(1)  The  condition  may  cause  death,  severe  injury,  severe  occupational  illness, 
or  major  system  damage  or  loss. 

(2)  The  condition  causes  unacceptable  delays  in  accomplishing  testing  or  prevents 
successful  mission  accomplishment  (due  to  severity  and  frequency  of  the  deficiency)  and 
would  critically  impact  the  operational  capability  of  the  system. 

b.  Category  II: 

(1)  Deficiency.  A  condition  which  prevents  successful  mission  accomplishment 
(system  does  not  meet  minimum  operational  requirements,  but  does  not  justify  immediate 
corrective  action  in  accordance  with  Cat  I)  or  degrades  a  system’s  operational  effectiveness 
and/or  suitability. 

(2)  Enhancement.  A  condition  that  would  complement  but  is  not  absolutely 
required  for  successful  mission  accomplishment.  The  recommended  condition,  if 
incorporated,  will  improve  a  system’s  operational  effectiveness  and/or  operational  suitability. 
Appendix  C  identifies  these  SRs  by  preceding  the  SR  number  with  the  letter  "e." 
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Table  !V-1  (continued) 


c.  Identified  During  IOT&E(2).  SRs  that  the  test  team  discovered  and  validated 
during  IOT&E(2).  Table  IV-2,  table  IV-3,  and  appendices  B  and  C  identify  these  SRs  by 
having  a  three-digit  SR  number. 

d.  Revalidated  During  IOT&E(2).  SRs  that  the  test  team  discovered  during 
IOT&E(1A)  and  IOT&E(1B)  and  revalidated  during  !OT&E(2).  Table  IV-2,  table  IV-3,  and 
appendices  B  and.  C  identify  these  SRs  by  having  a  three-digit  SR  number  followed  by 
the  letter  "A"  or  "B." 

4.1  PRIORITIZED  SRs.  The  test  team  prioritized  all  identified  and  revalidated  SRs  using 
the  Deficiency  and  Enhancement  Analysis  Ranking  Technique  method.  Table  IV-2  contains 
the  prioritized  list  of  the  Category  I  SRs,  and  table  IV-3  contains  a  prioritized  list  of  the 
top  40  Category  II  SRs  that  impacted  the  test  objectives.  All  655  SRs  are  included  in 
appendices  3  and  C.  The  complete  prioritized  list  of  12  Category  I  SRs  is  in  appendix 
B,  and  the  643  Category  II  SRs  are  in  appendix  C. 


Table  IV-2 


List  of  Prioritized  Category  I  Service  Reports 
Opened  During  IOT&E(2)  or  Revalidated  from  !OT&E(1A)  and  IOT&E(1B) 


Rank  SR# 


1 

168 

2 

Cl  2 

3 

010 

4 

26 4 A 

5 

262A 

6 

190 

7 

189 

8 

Oil 

o 

009 

10 

076 

11 

049 

12 

061 A 

Title 

Safety  -  Unsafe  Power  Down  Procedures  for  Component  Replacement 
Safety  -  Hazards  Associated  with  the  Use  of  Radome  Davit  Assembly 
Safety  -  Hazards  with  Large  Radome  Hatch  Cover 
Personnel  Hazard  Due  to  Potentially  Unprotected  Hatch  Opening 
Personnel  Safety  Hazard  When  Opening  the  RDA  or  RPG  Tower  Room 
Floor  Hatch 

Safety  -  "Eye  Wash"  Required  in  RDA  Generator  Shelter 

Safety  -  Hazard  Associated  With  Exhaust  Fan  in  Generator  Shelter 

Safety  -  Inadequate  Safety  Railing  Around  Large  Radome  Hatch  Opening 

Safety  -  Hazard  Associated  with  Entry/Exit  Radome  Hatch  Opening 

Safety  -  Inadequate/Inappropriate  Fire  Suppression  Systems  at  IOT&E(2) 

Principal  User  Processor  (PUP)  Sites 

Safety  -  Generator  Shelter  Entrance  Hazard 

Unusable  Handrail  in  Tower 
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Table  IV-3 

• 

List  of  Top  40  Prioritized  Category  II  Service  Reports 

Opened  During  IOT&E{2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B)  I 

• 

Rank 

SR  # 

Title 

1 

098B 

Potential  Transmitter  Reliability  Problem 

2 

531 

Too  Many  System  Status  False  Alarms 

3 

317 

Transfer  Between  Commercial  and  Backup  Power  Frequently  Forces 

The  RPG  into  an  Inoperable  Condition 

4 

206 

Spare  Transmitter  Line  Replaceable  Units’  (LRUs)  Configuration  Not 

Compatible  with  System  Under  Test 

5 

083 

Frequent  Ramtek  Graphics  Processor  Lock-Ups 

6 

072J 

Undefined  RDA  Alarms 

7 

006B 

Erroneous  Failure  Messages  on  System  Status  Menu 

8 

087B 

Failure  of  Automatic  RPG  Restart 

9 

014 

Chapter  5  Preliminary  Technical  Manual  (PTM)  Inadequate 

10 

265A 

Preliminary  Technical  Manual  Deficiencies 

11 

129 

Inadequate  Documentation  of  System  Status  Messages  at  the  RDA, 

UCP,  and  PUP  Applications  Terminals 

12 

530 

Deletion  of  and  Difficulty  in  Viewing  System  Status  Messages  at  UCP 
and  PUP 

13 

01 OA 

Loss  of  Radar  Data 

— 

14 

396 

New  Correction  Factors  for  Suncheck  Measurement  Subtest  1  (Align 

• 

15 

168A 

Pedestal)  Will  Not  Update  Correctly 

Audio  Alarms  at  the  UCP 

16 

437 

Numerous  PUP  Deficiencies  Apparently  Related  to  Graphic  Display  of 

Status  Messages 

17 

219 

Degraded  Operational  Utility  of  Base  Velocity  Products  Due  to  Range 

Folding 

18 

002B 

Transmitter  Faults  Causing  Wedges  of  Missing  Data 

19 

440 

Numerous  Discrepancies  in  "RDA  Calibration"  Procedures 

20 

164A 

Calibration  of  NEXRAD  Unit 

21 

400 

Corrupted  Links  In  Database  File 

22 

112A 

Failed  Power  Transistor 

23 

138A 

Undefined  PUP  System  Status  Messages 

24 

166 

System  Does  Not  Stay  On  Auxiliary  Power  When  Switchover  is 

Commanded  From  the  UCP  or  the  RDA  Maintenance  Terminal 

25 

017 

Orderly  Shutdown  of  RDA  at  RDA  Shelter  Not  Possible 

26 

391 

Safety  -  Inadequate  Waming/Caution  Signs  Throughout  NEXRAD 

- 

27 

227 

NEXRAD  Transmitter  Field  Maintenance  Manual  (NWS  EHB  6-514) 

Inadequate 

28 

500 

Apparent  Velocity  Dealiasing  Errors 

29 

196 

Recenter/Magnify  Product  Function  Unreliable 

30 

328 

NEXRAD  Commercial  Manuals  of  The  Preliminary  Technical  Manual 
(PTM)  Inadequate 

31 

384 

Inadequate  Procedures  in  RDASOT  User’s  Guide  for  Generation  of 

Clutter  Map 

32 

463 

Safety  -  Personnel  Hazard  Associated  With  the  Fixed  Ladder  Attached 
to  the  Antenna 

# 

33 

027B 

Need  for  Audio  Alarm  for  Free  Text  Message  (FTM) 

34 

138 

Chapters  1-4  and  6  of  Preliminary  Technical  Manual  (PTM)  Inadequate 

'-3 

Table  IV-3  (continued) 


List  of  Top  40  Prioritized  Category  II  Service  Reports 
Opened  During  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 


Rank 

SR  # 

35 

286 

36 

009A 

37 

420 

38 

207 

39 

087 

40 

133 

Title 

Safety  -  Inadequate  Warnings  Located  in  Chapter  5,  Preliminary  Technical 
Manual  (PTM) 

Need  for  Alert  of  System  Failure 

RDA  Transmitter  Beam  Voltage  Calibration  Data  Not  Available 
Safety  -  Inappropriate  Method  to  Bypass  Interlock  Switch  S4  in 
Transmitter  Cabinet 

Excessive  Acoustic  Noise  Associated  With  PUP  Cabinets 
Safety  -  Noncompliant  Grounding  and  Bonding 
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SECTION  V  -  SUMMARY  OF  CONCLUSIONS  AND  RECOMMENDATIONS 


5.0  SUMMARY: 

a.  A  matrix  of  test  results  for  those  objectives  for  which  operational  requirements 
existed  is  included  in  appendix  A. 

b.  The  definitions  of  the  terms  "evaluate,"  "assess,"  "met  requirement,"  and  "did  not 
meet  requirement"  are  contained  in  the  glossary  at  appendix  E. 

5.1  OBJECTIVE  E-1.  Evaluate  NEXRAD  as  an  effective  aid  in  preparing  accurate  and 
timely  weather  warnings.  (Reference  paragraph  3.1) 

5.1.1  Conclusion.  NEXRAD  met  the  users'  requirement  for  weather  warning  support. 
Operators  stated  that  NEXRAD  met  their  minimum  operational  requirements  primarily 
because  of  the  high  resolution  and  accuracy  of  the  NEXRAD  reflectivity-based  products. 
The  capability  to  magnify  and  time-lapse  storms  in  high  color  resolution,  the  use  of 
background  maps,  and  the  use  of  the  reflectivity-based  VIL  products  were  particularly 
effective.  However,  during  widespread  convective  activity,  velocity-based  products  were 
often  severely  degraded  by  large  areas  of  range-folded  and  incorrectly  dealiased  data. 
Additionally,  the  incorrectly  dealiased  velocity  fields  and  the  current  state  of  the 
mesocyclone  detection  and  hail  algorithms  resulted  in  numerous  false  severe  weather 
indications.  Further,  the  DOC  operators  could  not  locate  severe  storms  with  respect  to 
Oklahoma  cities  and  towns  with  the  contractor-provided  background  maps.  To  overcome 
this  deficiency,  DOC  operators  developed  a  city  background  map  of  sufficient  detail  to 
prepare  accurate  weather  warnings. 

5.1.2  Recommendations.  For  JSPO: 

a.  Eliminate  the  impact  of  range-folded  data  on  the  velocity-based  products  (SR  219). 

b.  Provide  an  effective  velocity  dealiasing  algorithm  (SRs  500.  062B,  441,  507). 

c.  Provide  reliable  hail  and  mesocyclone  detection  algorithm  outputs  (SRs  208.  380. 
228A). 

d.  Provide  complete  background  maps  with  adequate  detail  (SRs  050,  349,  238A. 
281,  122,  433). 

5.2  OBJECTIVE  E-2.  Evaluate  NEXRAD’s  impact  on  operator  workload.  (Reference 
paragraph  3.2) 

5.2.1  Conclusion.  NEXRAD  met  the  users’  requirement  for  operator  workload  when  the 
NEXRAD  PUP  alone  was  used  to  perform  existing  agency  weather  support  activities.  The 
operator  workload  when  the  UCP  and  PUP  were  used  together  did  not  meet  the  users’ 
requirement.  DOC  and  DOD  operators  found  that  using  the  NEXRAD  PUP  alone  to  meet 
existing  agency  requirements  resulted  in  a  slight  increase  in  operator  workload.  However. 
DOT  operators  stated  that  manually  acquiring  and  examining  products  from  multiple 
NEXRADs  would  produce  a  significant  increase  in  their  workload.  Operators  stated  that 
using  the  UCP  and  the  PUP  together  produced  a  significant  increase  in  operator  workload. 
Operators  noted  that  this  was  primarily  the  result  of  required  system  responsibilities  to 
support  the  multiple-user  radar  configuration.  These  UCP  duties  were  sometimes  delayed 
or  not  performed  because  of  other  mission  requirements.  Because  of  limitations  associated 
with  the  FTM  functionality,  operators  at  the  UCP  site  were  frequently  interrupted  from 
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mission  duties  to  respond  to  telephone  calls  from  the  other  two  associated  PUP  sites  or 
to  initiate  calls  to  them. 

5.2.2  Recommendations.  For  JSPO: 

a.  Correct  deficiencies  with  the  UCP  user  interface  (SRs  530,  168A,  009A,  167,  402, 
173,  082B,  337,  164,  069,  177,  175,  174). 

b.  Correct  deficiencies  associated  with  the  FTM  functionality  (SRs  027B,  178,  446, 
447,  445). 

c.  Correct  deficiencies  associated  with  the  dialup  interface  (SRs  395,  394,  515). 

5.3.  OBJECTIVE  E-3.  Assess  whether  current  position  qualifications  for  agency  personnel 
are  adequate  to  effectively  use  NEXRAD.  (Reference  paragraph  3.3) 

5.3.1  Conclusions.  There  was  a  consensus  among  operators,  supervisors,  and  specialists 
of  all  three  agencies  that  current  agency  position  qualifications  were  adequate  for  NEXRAD. 
However,  they  stated  that  without  proper  training,  personnel  having  these  qualifications  will 
not  be  able  to  use  NEXRAD  PUPs  and  UCPs  effectively  for  the  duties  of  their  assigned 
positions  (e.g.,  meteorologist,  weather  officer,  weather  forecaster,  or  weather  observer). 

5.3.2  Recommendation.  For  JSPO  and  users:  Ensure  effective  and  appropriate  NEXRAD 
operations  training  is  provided  to  agency  personnel  (also  see  objective  ES-9,  Training). 

5.4  OBJECTIVE  E-4.  Evaluate  NEXRAD  capability  to  provide  required  operational  support 
to  multiple  users.  (Reference  paragraph  3.4) 

5.4.1  Conclusions.  N EXRAD’s  capability  to  provide  required  operational  support  to  multiple 
users  met  the  users’  requirement.  For  associated  users,  operators  from  the  three  test  sites 
stated  that,  in  general,  NEXRAD  provided  RPS  products  in  a  timely  manner  including  times 
when  the  unit  was  operated  in  a  mode  simulating  a  19-user  configuration.  Because  of 
their  reliance  on  cross-section  and  WER  products  during  the  test  and  dialup  feature 
limitations,  the  DOT  operators  stated  that  the  responsiveness  of  the  NEXRAD  system  did 
not  meet  their  operational  requirements.  For  nonassociated  users,  operators  and  specialists 
identified  several  deficiencies.  First,  dialup  procedures  for  acquiring  products  from  multiple 
RPGs  were  cumbersome  and  time-consuming.  This  deficiency  will  particularly  impact 
agency  centers  requiring  routine  access  to  multiple  RPGs.  Second,  the  RPG  telephone 
number  directory  could  only  contain  a  maximum  of  12  digits  for  each  RPG,  making  long¬ 
distance  dialing  through  most  facility  switchboards  impossible.  Third,  background  maps 
from  nonassociated  RPGs  were  automatically  deleted  from  the  PUP  product  data  base  after 
only  6  hours.  In  addition,  the  functionality  to  store  and  retrieve  maps  using  optical  disk 
media  was  inoperable.  Therefore,  operators  had  to  repeatedly  request  maps  over  dialup 
lines.  Site  supervisors  found  that  the  URC  was  an  effective  forum  for  the  principal  user 
agencies  to  coordinate  the  use  of  NEXRAD.  Test  team  specialists  noted  that  URC- 
developed  agreements  need  to  be  quickly  incorporated  into  each  station's  operating 
procedures.  Further,  the  specialists  identified  the  need  for  strong  agency  support  and 
guidance  regarding  multiple  user  support  functions  and  how  NEXRAD-related  responsibilities 
relate  to  current  duty  priorities. 
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5.4.2  Recommendations.  For  JSPO: 

a.  Provide  an  effective  capability  to  acquire  products  from  multiple  RPGs  (SRs  395, 
393,  394,  515). 

b.  Provide  the  capability  to  retain  nonassociated  background  maps  in  a  separate 
PUP  storage  area  (SR  410). 

c.  Provide  the  capacity  tc  store  and  retrieve  nonassociated  RPG  background  maps 
(SR  338,  326). 

d.  Investigate  the  adequacy  of  NEXRAD  to  support  receipt  of  products  during  periods 
of  widespread  precipitation  (SR  502). 

e.  Ensure  cross-section  and  WER  products  are  received  in  a  timely  manner. 

5.4.2.2  For  users: 

a.  Develop  procedures  for  responsive  implementation  of  URC-coordinated  changes 
at  individual  associated  site  locations. 

b.  Provide  guidance  regarding  multiple-user  support  functions  and  how  NEXRAD- 
related  responsibilities  relate  to  current  duty  priorities. 

5.5  OBJECTIVE  E-5.  Evaluate  NEXRAD  as  an  effective  aid  in  preparing  accurate  and 
timely  weather  advisories.  (Reference  paragraph  3.5) 

5.5.1  Conclusions.  NEXRAD  met  the  users’  requirement  for  weather  advisories. 
Operators  reported  that  the  resolution  of  the  reflectivity  products  allowed  them  to  accurately 
identify  the  location  of  significant  weather  with  respect  to  specific  advisory  and  aircraft  route 
locations.  The  sensitivity  of  the  reflectivity  products  allowed  operators  to  identify  many 
features  such  as  gust  fronts,  thunderstorm  outflow  boundaries,  and  fine  lines.  Identification 
of  these  features,  combined  with  the  use  of  VAD  wind  profiles  and  base  velocity  products 
to  identify  inversions  and  low-level  jet  streams,  enabled  operators  to  provide  timely  terminal 
wind  advisories  and  low-level  wind  shear  advisories.  However,  operators  did  not  find  useful 
information  in  the  layered-turbulence  products.  Previously  identified  deficiencies  with 
velocity  dealiasing  and  range-folded  velocity  data  often  prevented  operators  from 
determining  the  strength  of  winds  associated  with  convective- related  features  identified  in 
the  reflectivity  data  (e.g.,  gust  fronts).  The  test  team  noted  two  limitations  associated  with 
the  use  of  the  automated  alert  feature  that  reduced  this  feature's  effectiveness.  First,  the 
current  state  of  the  storm-series  algorithms  appeared  to  produce  frequent  false  indications 
of  significant  weather  (e.g.,  hail  and  mesocyclonic  shear).  Second,  specialists  observed 
that  because  of  inadequate  applications  training  operators  did  not  always  know  how  to 
apply  alert  thresholds  and  alert  areas  to  match  existing  meteorological  conditions. 

5.5.2  Recommendations. 

5.5.2.1  For  JSPO: 

a.  Provide  effective  layered  turbulence  products  (SRs  160,  421). 

b.  Eliminate  the  impact  of  range-folded  data  on  the  velocity-based  products  (SR  219). 


V-3 


c.  Provide  an  effective  velocity  dealiasing  algorithm  (SRs  500,  062B.  441). 

d.  Provide  effective  haif  and  mesocyclone  algorithms  (SRs  380,  228 A). 

5.5.2.2  For  JSPO  and  users:  Provide  adequate  training  on  the  appropriate  application 
of  the  automated  alert  feature  for  each  users’  weather  support  requirements  (SRs  01 5,  247, 
455). 

5.6  OBJECTIVE  E-6.  Evaluate  NEXRAD  as  an  effective  aid  in  providing  routine  weather 
services.  (Reference  paragraph  3.6) 

5.6.1  Conclusions.  NEXRAD  met  the  users’  minimum  requirement  as  an  effective  aid  in 
short-range  forecasts,  surface  observations,  briefings,  and  aircraft  traffic  management.  To 
support  routine  weather  services,  the  high  resolution  and  sensitivity  of  NEXRAD  aided  in 
the  identification  of  fronts,  wind  shift  lines,  precipitation  areas,  and  dry  lines.  Clear-air 
mode  operation  was  particularly  effective  in  identifying  small-scale  features.  VAD  and 
base  velocity  products,  when  not  contaminated  by  large  areas  of  range-folded  and 
incorrectly  dealiased  data,  aided  in  the  preparation  of  surface  forecasts  and  in  diagnosing 
vertical  wind  field  changes.  DOD  observers  stated  they  could  effectively  use  NEXRAD 
products  and  manipulation  features  to  determine  storm  location  and  movement  for  inclusion 
in  surface  weather  observation  remarks.  DOD  forecasters  and  observers  stated  they  were 
able  to  prepare  a  reflectivity-only  radar  observation  more  accurately  and  typically  in  less 
than  half  the  time  with  NEXRAD  than  is  presently  required  for  the  FPS-77  weather  radar. 
DOD  forecasters  stated  the  ability  to  time-lapse  color  radar  information  and  remote  that 
information  to  the  briefing  counter  was  particularly  valuable.  DOC  operators  were  able  to 
prepare  civil  defense  briefings  primarily  because  of  detailed  reflectivity  data  placement  on 
the  country  and  operator-generated  city  background  maps.  DOT  and  DOD  operators  noted 
the  effectiveness  of  the  reflectivity  and  VAD  products  aided  in  displaying  the  meteorological 
conditions  for  planned  briefings.  However,  the  DOT  operators  stated  that  on-demand 
briefing  effectiveness  was  degraded  because  one-time  and  dialup  product  requests  were 
not  responsive  (see  objective  E-4).  Additionally,  the  automatic  scan  mode  deselection 
feature  often  forced  an  operationally  undesirable  switch  to  the  precipitation  mode  because 
of  AP.  All  DOC  operators  stated  that  the  new  requirement  of  editing  the  RCM  produced 
a  significant  increase  in  their  workload.  Operators  spent  significant  time  verifying  and 
removing  residual  clutter,  AP,  and  false  indications  of  mesocyclones  and  hail. 

5.6.2  Recommendations. 

5.6.2.1  For  JSPO: 

a.  Eliminate  the  impact  of  range-folded  data  on  the  velocity-based  products  (SR  219). 

b.  Provide  an  effective  velocity  dealiasing  algorithm  (SRs  500,  062B,  441). 

c.  Reduce  the  impact  of  the  RCM  on  operator  workload  (SRs  484,  258A,  307,  358, 
411,  333,  427,  336,  385). 

d.  Ensure  one-time  products  are  received  in  a  timely  manner  for  on-demand  briefings 
(SR  166  A). 

e.  Provide  an  effective  capability  to  acquire  products  from  multiple  RPGs  (SRs  395. 
393,  394,  515). 
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5.6.2.2  For  JSPO  and  users:  Provide  the  UCP  operator  the  capability  to  override  the 
automatic  scan  mode  deselect  feature  and  1-hour  timeout  when  operationally  required. 
Ensure  the  FMH-1 1  allows  the  UCP  operator  to  use  this  capability.  (SR  250A). 

5.7  OBJECTIVE  E-7.  Assess  NEXRAD  as  an  effective  aid  to  meeting  agency  mission 
requirements  when  changing  to.  operating  on,  and  recovering  from  backup  power. 
(Reference  paragraph  3.7) 

5.7.1  Conclusions.  NEXRAD  was  not  an  effective  aid  in  meeting  agency  mission 
requirements  when  changing  to  and  recovering  from  backup  power.  During  IOT&E(2)  Part 
B,  the  system  failed  17  times  (RDA  4  times,  RPG  12  times,  and  PUP  1  time)  because 
of  power  transitions-whether  unscheduled  or  operator-initiated.  In  these  cases,  a 
maintenance  action  and  a  manual  restart  was  required.  Outage  times  resulting  from  power 
transfers  ranged  from  11  minutes  to  8  hours  54  minutes.  These  failures  resulted  in  an 
increase  in  workload,  an  increase  in  maintenance  interventions,  and  the  loss  of  critical 
radar  data.  Operators  stated  that  the  loss  of  critical  radar  data  during  significant  weather 
situations  resulted  in  a  significant  decrease  in  the  effectiveness  of  NEXRAD  as  an  aid  in 
providing  weather  warning  and  advisory  support.  Conversely,  operators  observed  that  the 
three  operational  PUPs  recovered  automatically  after  power  transitions  except  for  one  event 
at  Tinker  AFB  BWS.  Operators  did  not  observe  any  change  in  system  performance  or 
operator  workload  when  NEXRAD  was  operating  on  backup  power. 

5.7.2  Recommendation.  For  JSPO:  Ensure  the  RDA  and  RPG  effectively  and 
automatically  return  to  an  operational  state  following  power  transitions  (SRs  317,  087B). 

5.8  OBJECTIVE  E-8.  Assess  NEXRAD  electromagnetic  compatibility  (EMC).  (Reference 
paragraph  3.8) 

5.8.1  Conclusions.  The  test  team  maintenance  technicians  noted  one  apparent  EMC 
problem--a  wavy  presentation  on  the  RDA  applications  terminal  throughout  IOT&E(2). 
Operators  did  not  observe  any  EMC  incidents  associated  with  the  operation  of  NEXRAD 
equipment,  nor  was  there  any  observable  effect  on  any  nearby  equipment. 

5.8.2  Recommendation.  For  JSPO:  Investigate  and  resolve  cause  of  the  RDA 

applications  terminal  having  a  wavy  presentation  (SRs  051  A,  131). 

5.9  OBJECTIVE  ES-9.  Assess  the  adequacy  of  the  planned  NEXRAD  training  to  provide 
the  skills  required  to  effectively  use  and  maintain  NEXRAD.  (Reference  paragraph  3.9) 

5.9.1  Conclusions.  The  training  assessment  was  separated  into  three  areas:  operations, 
maintenance,  and  software. 

5.9.1. 1  Operations.  Test  team  training  specialists  stated  that  the  planned  Cadre  and 
Interim  Operations  courses  were  deficient.  The  Cadre  course  lacked  sufficient  detail  and 
would  not  adequately  prepare  agency  instructors  to  teach  NEXRAD  operations.  In  addition, 
they  stated  the  Interim  Operations  Course  would  not  support  the  training  of  students  to 
the  agency-required  skill  level.  DOC  training  specialists  stated  that  DOC/DOT  planned 
CBT  was  an  area  of  high  risk.  Training  specialists  identified  deficiencies  associated  with 
the  planned  CBT,  and  current  DOC  plans  did  not  address  required  on-site  training.  A 
comprehensive  DOD  NEXRAD  operations  training  plan  had  not  been  prepared.  Formal 
DOD  course  requirements  had  not  been  finalized:  consequently,  manpower  requirements 
to  support  training  had  not  been  adequately  defined.  The  PUP  training  mode  and  NEXRAD 
archive  functionality  demonstrated  a  potential  to  support  hands-on  operations  training. 
However,  test  team-identified  deficiencies  with  these  features  limited  their  usefulness  during 
IOT&E(2). 
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5.9.1. 2  Maintenance.  Without  significant  changes,  the  planned  NEXRAD  maintenance 
training  will  not  provide  the  necessary  training  for  an  agency  technician  to  acquire  the 
needed  skills  to  effectively  maintain  a  NEXRAD  system  in  accordance  with  the  maintenance 
concept.  The  test  team  identified  several  deficiencies  with  planned  maintenance  training. 
In  addition,  deficiencies  were  identified  in  the  7-week  IOT&E(2)  maintenance  course.  The 
PTM,  which  was  used  as  the  primary  course  reference,  was  ineffective  as  a  training  tool 
(see  objective  S-15).  These  training  deficiencies  directly  contributed  to  the  excessive 
troubleshooting  and  repair  times  experienced  during  IOT&E(2).  Unless  these  training 
problems  are  resolved  before  the  start  of  cadre  training,  the  goals  of  the  NEXRAD 
maintenance  concept  will  not  be  achieved  and  operational  availability  will  probably  be 
adversely  affected. 

5.9.1 .3  Software.  The  planned  NEXRAD  training  will  probably  not  provide  the  skills 
necessary  to  effectively  maintain  the  NEXRAD  software.  The  structure  of  the  course  did 
not  follow  an  organized,  logical  plan.  The  course  provided  an  adequate  knowledge  of  the 
organization  and  operation  of  NEXRAD  software  but  not  the  detailed  skills  and  procedures 
needed  for  software  maintenance.  There  was  insufficient  hands-on  laboratory  time  to  gain 
experience  with  the  NEXRAD  software  maintenance  procedures.  A  review  of  the  proposed 
14-week  course  showed  that  the  same  deficiencies  identified  in  the  7-week  IOT&E(2) 
course  will  probably  be  repeated.  In  addition,  no  follow-on,  OJT,  or  additional  formal 
training  was  planned.  Unless  the  deficiencies  identified  above  are  corrected,  software 
maintainers  will  likely  require  an  extensive  on-the-job  trial-and-error  process  to  acquire  the 
skills  needed  to  maintain  the  NEXRAD  software. 

5.9.2  Recommendations. 

5.9.2.1  For  JSPO: 

a.  Operations: 

(1)  Ensure  adequacy  of  Personnel  Requirements,  Training,  and  Training 
Equipment  Plan  (CDRL  218)  in  meeting  agency  operations  training  requirements. 

(2)  Correct  deficiencies  associated  with  the  PUP  training  mode  (SRs  059.  162. 
460,  579,  505). 

(3)  Correct  deficiencies  associated  with  NEXRAD  archive  functionality  to  help 
support  operator  training.  (SRs  120,  325,  351,  194,  338). 

b.  Maintenance.  Ensure  the  technical  manuals  are  sufficiently  upgraded  and 
adequate  course  material  is  developed  to  meet  both  the  theory  and  hands-on  training 
requirements  of  the  cadre  training  course  and  the  first  increment  of  field  maintainers. 
(SRs  014,  265A,  129,  440,  138A,  227,  328,  384,  138,  286,  420,  544,  251). 

c.  Software: 

(1)  Ensure  the  contractor's  14-week  software  maintenance  course  is  restructured 
to  follow  a  logical,  organized  plan. 

(2)  Ensure  the  focus  and  level  of  detail  of  the  contractor’s  14-week  software 
maintenance  course  provide  the  students  instruction  in  the  proper  use  of  the  software 
tools  necessary  to  maintain  the  NEXRAD  software. 

f3)  Ensure  the  contractor’s  14-week  software  maintenance  course  provides 
adequate  hands-on  laboratory  time. 
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5.9.2.2  For  JSPO  and  users  -  Maintenance: 


a.  Ensure  the  contractor's  maintenance  instructors  are  sufficiently  knowledgeable  of 
NEXRAD  to  teach  both  theory  and  hands-on  maintenance  for  all  functional  areas.  Until 
Unisys  demonstrates  the  ability  to  provide  an  adequate  training  course,  make  maximum 
use  of  subcontractor  equipment  training  experts  (e.g.,  Concurrent  Computer  Corporation 
training  instructors). 

b.  Ensure  detailed  lesson  plans  are  developed  well  in  advance  of  cadre  training. 
Inspect  these  plans  to  determine  adequacy  of  course  content  and  length. 

c.  Ensure  the  course  contains  an  introduction  to  all  areas  of  instruction  that  have 
not  been  previously  taught  to  current  5-level  technicians  (e.g.,  fiber  optics,  computer 
architecture,  etc.). 

5.9.2.3  For  DOC  and  DOT  -  Operations: 

a.  Evaluate  the  potential  of  supplementing  CBT  instruction  at  the  training  site  with 
hands-on  use  of  PUPs  and  an  RPG  using  Archive  II  playback  capability. 

b.  Prepare  training  materials  to  address  on-site,  follow-on  NEXRAD  training. 

5.9.2.4  For  DOD  -  Operations: 

a.  Ensure  a  comprehensive,  coordinated  training  plan  is  developed. 

b.  Ensure  manpower  requirements  to  meet  training  needs  are  adequately  defined 
and  personnel  are  available  in  time  to  prepare  for  cadre  training. 

5.9.2.5  For  OSF: 

a.  Ensure  adequate  OJT  materials  and  a  follow-on  software  maintenance  course 
are  developed  for  training  OSF  software  personnel. 

b.  Ensure  adequate  system  time  is  provided  for  operations,  maintenance,  and 
software  course  development  and  for  hands-on  instruction  during  laboratory  sessions. 

5.10  OBJECTIVE  ES-10.  Assess  impacts  of  any  safety  hazards  associated  with  NEXRAD. 
(Reference  paragraph  3.10) 

5.10.1  Conclusions.  The  test  team  identified  and  documented  56  safety  deficiencies 
during  IOT&E(2).  Nine  of  the  deficiencies  were  hazards  that  had  the  potential  to  cause 
death,  severe  injury,  and/or  major  system  damage  (Category  I).  Of  the  remaining  47  safety 
deficiencies  (Category  II),  16  had  the  potential  to  cause  minor  injury  to  personnel  or  minor 
damage  to  the  equipment,  while  the  other  9  had  the  potential  to  cause  minor  equipment 
damage  only.  All  identified  safety  deficiencies  were  documented  in  service  reports. 

5.10.2  Recommendations.  For  JSPO: 

a.  Ensure  the  contractor  corrects  ail  identified  Category  I  safety  deficiencies. 

(SRs  168,  012,  010,  264A,  262A,  190,  189,  011,  009,  076,  049,  061  A) 

b.  Ensure  the  contractor  corrects  all  identified  Category  II  safety  deficiencies 

(SRs  391,  463,  286,  207,  133,  404,  169,  357,  533,  032,  098,  113) 
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c.  Ensure  safety  warnings  and  safe  equipment  power-down/power-up  instructions  are 
incorporated  into  ail  applicable  maintenance  procedures  in  the  technical  data.  (SRs  168, 
286,  285) 

5.11  OBJECTIVE  ES-11.  Assess  factors  impacting  the  interoperability  of  NEXRAD  with 
existing  and  planned  systems.  (Reference  paragraph  3.11) 

5.1 1 .1  Conclusions.  The  test  team  found  there  was  inadequate  information  in  the  interface 
control  documents  to  interface  planned  systems  with  PUES  communications  ports  using 
the  Redbook  data  formats.  Information  in  the  CIUG  and  ICDs  was  not  logically  ordered, 
and  topics  were  scattered  over  several  documents.  Although,  there  appeared  to  be 
sufficient  information  to  interface  systems  with  the  "Other  Users"  ports  on  NEXRAD,  test 
team  specialists  identified  deficiencies  that  made  it  difficult  and  time-consuming  to  find  and 
organize  the  required  information. 

5.11.2  Recommendations. 

5.11.2.1  For  JSPO: 

a.  Provide  a  stand-alone  interface  document  for  each  NEXRAD  interface.  (SRs  407, 
526,  302) 

b.  Clearly  document  deviations  from  accepted  standards  and  protocols  (SRs  261, 
486). 

5.11.2.2  For  users:  Investigate  if  the  identified  concerns  associated  with  the  PUES  port 
will  adversely  impact  its  intended  use. 

5.12  OBJECTIVE  S-12.  Assess  NEXRAD  reliability.  (Reference  paragraph  3.12) 

5.12.1  Conclusions.  The  demonstrated  MTBM  (total  corrective)  for  the  NEXRAD  system 
was  25.3  hours.  The  demonstrated  MTBM  (total  corrective)  for  the  RDA.  RPG.  and  PUP 
was  53.1  hours,  78.6  hours,  and  125.6  hours,  respectively.  Reliability  problems  were 
identified  with  the  preproduction  transmitter,  the  RPG  following  power  transitions,  the 
graphics  processors,  and  the  optical  disk  drive  unit.  The  decreased  reliability  and  the 
similar  maintainability  between  current  agency  weather  radars  and  NEXRAD  indicated  that 
NEXRAD  will  increase  the  workload  for  technicians  at  maintenance  locations  responsible 
for  an  entire  NEXRAD  system.  For  PUP  only  sites,  NEXRAD  may  have  little  or  no  impact 
on  maintenance  workload. 

5.12.2  Recommendations.  For  JSPO: 

a.  Assess  transmitter  reliability  and  take  appropriate  action  to  correct  recurring 
transmitter  problems.  (SRs  098B,  002B,  112A,  149) 

b.  Correct  problems  associated  with  the  RPG  failing  to  recover  automatically  after 
power  transfers.  (SRs  31 7,  087B) 

c.  Determine  the  underlying  causes  of  Ramtek  graphics  problems  and  take 
appropriate  action  to  eliminate  recurrence.  (SRs  083,  301,  418) 

d.  Correct  problems  associated  with  the  optical  disk  drive  unit  and  archive 
functionality.  (SRs  368,  061,  332.  051) 

e.  For  all  other  failures,  determine  the  failure  sources  and  take  corrective  action. 


5.13  OBJECTIVE  S-13.  Evaluate  NEXRAD  maintainability.  (Reference  paragraph  3.13) 

5.13.1  Conclusions.  For  LRU  malfunctions,  the  demonstrated  system  MTTR  of  9.0  hours 
did  not  meet  user’s  requirement  of  0.5  hour.  PFI  isolated  50  percent  of  LRU  malfunctions 
to  a  single  LRU,  which  did  not  meet  the  user’s  requirement  of  80  percent.  The  PFI 
isolated  LRU  malfunctions  to  three  or  fewer  LRUs  57.1  percent  of  the  time,  which  did  not 
meet  the  user's  requirement  of  95  percent.  The  three  primary  deficiencies  that  contributed 
to  the  system’s  MTTR  were  training  (see  objective  9),  the  PTM  (see  objective  15),  and 
PFI.  During  IOT&E(2),  PTM  fault  isolation  flowcharts,  on-line  diagnostics,  and  off-line 
diagnostics  were  inadequate  for  isolating  faults  in  the  NEXRAD  system  within  the 
maintenance  concept.  The  fault  isolation  flowcharts  had  limited  usefulness  as  the  primary 
fault  isolation  tool.  The  flowcharts  were  incomplete  and  ambiguous  and  they  contained 
numerous  errors.  The  on-line  diagnostic’s  use  of  BIT  and  self-diagnostic  logic  seemed 
to  be  sufficiently  integrated  within  the  system;  however,  several  deficiencies  limited  its 
benefit.  Off-line  diagnostics  were  not  sufficient  to  isolate  faults.  Adequately  detailed 
documentation  for  each  off-line  diagnostics  was  not  available.  The  system  MTTR  was 
greatly  impacted  by  the  RDA  MTTR  of  18.0  hours.  The  on-line  system  status  monitoring 
system  generated  status  alarms/messages  so  frequently  (approximately  45  per  hour)  that 
the  PUP  and  UCP  operators  often  ignored  them,  even  though  some  indicated  "maintenance 
mandatory." 

5.13.2  Recommendations.  For  JSPO: 

a.  Ensure  the  technical  data  are  adequate  to  maintain  the  system.  (SRs  014.  265A, 
129,  440,  138A.  227.  328,  384,  138.  286,  420,  544,  251,  543,  403,  239,  169A,  285,  416, 
439,  077) 

b.  Resolve  all  training  issues  impacting  maintainability  (see  objective  ES-9). 

c.  Ensure  on-line  fault  monitoring  is  improved  by  reducing  the  frequency  of  system 
status  alarms/messages  and  by  eliminating  unconfirmed  fault  indications.  (SRs  531 . 072B. 
006B,  129,  530,  168A.  437.  138A.  009A.  139A.  104A.  439) 

d.  Ensure  the  system  on-line  BIT  and  self-diagnostics  are  improved  to  consistently 
and  accurately  isolate  faults  within  specific  areas/subsystems.  (SRs  072B,  439,  21 2,  248 
300,  057.  541.  255A,  386,  467,  213) 

e.  Ensure  all  off-line  diagnostic  tests  are  improved  so  that  LRU  malfunctions  can 
be  isolated  within  the  criteria  specified  by  the  maintenance  concept.  (SRs  251,  169A,  264. 
018,  008,  378,  170,  048,  094,  064,  066,  117,  245,  151,  082,  141,  368,  065,  471.  319.  470. 
469) 


f.  Provide  sufficient  storage  and  workspace  for  maintenance  in  the  RDA  and 
generator  shelters.  (SRs  187,  047B,  096) 

g.  Ensure  all  equipment/LRUs  are  correctly  labeled  and  cable  routing  and  terminations 
are  designed  for  ease-of-maintenance.  (SRs  098,  159,  055,  208A,  124,  125A,  268.  036. 
132,  262,  312,  314,  038,  034B,  292,  347,  144,  472,  134,  232,  070B,  148,  099.  045,  030 
156,  145,  105,  054,  116,  171,  114) 

h.  Provide  secondary  fault  isolation  procedures  to  augment  and  back  up  the  PFI 
(SR  169A) 
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5.14  OBJECTIVE  S-14.  Evaluate  NEXRAD  availability.  (Reference  paragraph  3.14) 

5.14.1  Conclusions.  The  NEXRAD  full  system  operational  availability  of  86.3  percent  did 
not  meet  the  user’s  requirement  of  90  percent.  The  NEXRAD  degraded  system  operational 
availability  of  88.2  percent  did  not  meet  the  user’s  requirement  of  96  percent.  The  RDA 
availability  had  the  greatest  impact  on  system  availability.  The  largest  detractor  from  the 
RDA  availability  was  the  preproduction  transmitter  reliability  and  maintainability  problem 
(11  transmitter  failures  with  a  10.4  hour  MTTR).  Although  the  RPG  failures  associated 
with  power  transitions  did  not  have  a  significant  impact  on  overall  system  availability,  the 
timing  of  these  events  critically  degraded  operational  effectiveness.  When  the  operators 
considered  total  system  effectiveness,  including  the  impact  of  availability  for  supporting 
mission  requirements,  they  stated  that  the  system  did  not  meet  their  minimum  operational 
requirements. 

5. 1 4.2  Recommendations. 

5.14.2.1  For  JSPO: 

a.  Ensure  RDA  reliability  and  maintainability  problems,  particularly  with  the  transmitter, 
are  resolved.  (SRs  098B,  206,  002B,  112A,  420,  207,  354,  096,  026,  149,  036,  078,  033B. 
267,  483,  353,  185,  327,  073,  095,  118) 

b.  Ensure  RPG  power  transfer  problems  are  resolved.  (SRs  31 7,  087B,  1 66) 

c.  Ensure  overall  system  maintainability  problems  are  resolved  (see  objectives  S-12 
and  S-13). 

5.14.2.2  For  JSPO  and  users:  Ensure  sparing  level  is  adequate  to  meet  availability 
requirements. 

5.15  OBJECTIVE  S-1 5.  Assess  the  adequacy  of  logistics  support.  (Reference  paragraph 
3.15) 

5.15.1  Conclusions.  The  PTM  and  the  JSPO  list  of  required  support  equipment  did  not 
agree.  The  test  team  identified  20  items  of  support  equipment  required  by  the  PTM  for 
maintenance  activities  which  were  not  on  the  JSPO  list.  These  deficiencies  limited  the 
maintainers’  ability  to  perform  or  complete  scheduled  and  unscheduled  maintenance  actions. 
The  JSPO-provided  complement  of  on-site  spares  was  not  adequate  to  maintain  the 
system  in  accordance  with  the  maintenance  concept.  Therefore,  the  provisioning  process 
must  identify  a  sparing  level  that  is  better  aligned  to  the  agencies'  requirements  than  the 
contractor-proposed,  JSPO-approved  package  of  spares  that  was  provided  for  IOT&E(2). 
Of  the  56  LRU  replacements  required  during  IOT&E(2),  16  spares  (28.6  percent)  were 
on  site  and  the  remaining  40  spares  (71.4  percent)  had  to  be  ordered.  Also,  upon  receipt. 
1 1  of  the  40  contractor-provided  spares  were  incompatible  with  the  unit  being  tested  in 
IOT&E(2).  The  majority  of  the  incompatible  LRUs  were  for  the  RDA.  The  unanimous 
opinion  of  the  maintenance  technicians  was  that  the  PTM  was  inadequate  for  training  and 
for  maintaining  the  NEXRAD  system.  The  PTM  was  incomplete  and  ambiguous  and 
contained  numerous  errors.  As  a  result,  maintenance  technicians  stated  that  the  PTM 
was  inadequate  for  83  out  of  159  maintenance  actions  documented  on  maintenance 
incident  questionnaires  during  IOT&E(2).  Additionally,  of  the  72  PMIs  scheduled  during 
Part  B,  42  had  documented  technical  data  deficiencies:  17  of  these  42  could  not  be 
completed.  The  planned  cadre  training  will  likely  be  ineffective  if  the  technical  manuals 
do  not  have  major  upgrade  prior  to  its  start.  Until  the  technical  manuals  are  complete  and 
the  ambiguities  and  errors  are  removed,  the  NEXRAD  system  will  probably  not  be 
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maintainable  in  accordance  with  the  NEXRAD  maintenance  concept  and  operational 
effectiveness  will  likely  be  adversely  impacted. 

5. 1 5.2.  Recommendations. 

5.15.2.1  For  JSPO  and  users: 

a.  Ensure  the  technical  data  identify  all  support  equipment  required  to  complete 
organizational-level  maintenance.  (SRs  202,  078,  023,  483,  209,  192,  536,  107,  236) 

b.  Ensure  sparing  level  is  adequate  to  meet  availability  requirements. 

5.15.2.2  For  JSPO: 

a.  Ensure  spares  provided  by  depot  are  compatible  with  the  fielded  unit  (e.g.,  limited 
production  spares  for  limited  production  equipment).  (SRs  206,  480,  099) 

b.  Ensure  the  technical  data  are  significantly  upgraded,  validated,  and  verified  well 
before  the  cadre  training  to  meet  both  the  theory  and  hands-on  training  requirements. 
(SRs  014,  265A,  129,  440,  138A,  227,  328,  384,  138,  286,  420,  544,  251) 

c.  Ensure  all  alignments  and  PMI  procedures  that  are  required  to  maintain  the 
NEXRAD  system  are  correct  and  included  in  the  technical  data.  (SRs  440,  164A,  384, 
420,  169,  354,  285,  197,  355,  482,  005,  112,  412,  535,  312,  353) 

d.  Ensure  the  NEXRAD  system  technical  data  are  adequate  for  a  5-level  maintenance 
technician  to  maintain  NEXRAD  in  accordance  with  the  maintenance  concept. 

e.  Provide  secondary  fault  isolation  procedures  to  augment  and  back  up  the  PFI. 
(SR  169  A) 

5.16  OBJECTIVE  S-1 6.  Evaluate  NEXRAD  software  maintainability.  (Reference  paragraph 
3.16) 

5.16.1  Conclusions.  Overall,  the  documentation  and  source  listing  evaluation  for  the  four 
CPCIs  evaluated  met  the  users'  requirement  of  3.5. 

5.16.1.1  The  evaluators  identified  significant  problems  in  the  documentation.  The 
characteristics  of  testability  and  traceability  did  not  meet  the  requirement  of  3.5  for  any 
of  the  four  CPCIs.  The  characteristic  of  descriptiveness  did  not  meet  the  requirements 
of  3.5  for  two  of  the  four  CPCIs  evaluated.  Expandability  did  not  meet  the  requirement 
for  CPCI-01 .  The  primary  reasons  why  these  documentation  characteristics  average  below 
3.5  are  listed  below.  First,  the  C5s  were  inadequate  for  each  of  the  CPCIs  evaluated. 
It  was  difficult,  time-consuming,  and  sometimes  impossible  to  find  module  descriptions,  data 
flow  descriptions,  and  calling  sequences.  Second,  the  data  dictionaries  were  inadequate 
for  each  CPCI  evaluated.  Third,  the  VDD  for  each  CPCI  did  not  contain  adequate 
descriptions.  The  files  needed  to  compile  and  link  a  CPCI  were  not  fully  specified,  and 
other  CPCIs  associated  with  or  used  by  a  CPCI  were  inadequately  specified. 

5.16.1.2  The  source  listings  were  determined  to  have  simple,  expandable,  and  modular 
characteristics.  These  coding  characteristics  enhanced  the  maintainability  of  the  software. 
However,  the  characteristic  of  traceability  did  not  meet  the  requirements  for  any  of  the  four 
CPCIs  evaluated.  The  major  deficiency  that  adversely  impacted  traceability  was  the 
inadequate  preface  block  in  each  module’s  source  listing.  Of  the  179  modules  evaluated, 
165  modules  had  errors,  inconsistencies,  or  incomplete  information. 
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5.16.1.3  Although  the  documentation  and  source  listings  met  the  users’  requirements  as 
evaluated  using  standard  questionnaires,  additional  personnel  and  other  resources  will  likely 
be  necessary  to  maintain  and  update  the  N  EX  RAD  software  unless  the  above  identified 
deficiencies  are  corrected. 

5.16.2  Recommendations.  For  JSPO: 

a.  Ensure  the  contractor  reviews  and  corrects,  for  all  modules,  the  deficiencies 
associated  with  the  preface  blocks  and  imbedded  comments  of  the  source  listings.  (SRs 
352,  496,  497,  401) 

b.  Correct  the  deficiencies  with  the  software  C5  documentation.  (SRs  335,  493,  290, 
289,  492,  374,  491,  362,  342,  498,  334,  316) 

c.  Correct  the  deficiencies  with  the  data  dictionaries.  (SR  323) 

d.  Ensure  each  version  description  document  adequately  describes  each  CPCI. 
(SRs  494,  341) 

5.17  OBJECTIVE  S-17.  Assess  the  adequacy  of  planned  and  existing  N EXRAD  software 
support  resources  (SSR).  (Reference  paragraph  3.17) 

5.17.1  Conclusions.  There  is  a  risk  that  the  current  and  planned  software  support 
resources  may  not  be  adequate  for  the  government  to  assume  software  support 
responsibilities  at  the  appropriate  time.  The  OSF  was  hiring  personnel  with  adequate 
experience  and  skills  and  was  addressing  facility  space  shortfalls.  However,  detailed 
project  and  configuration  management  plans  were  incomplete  and  had  not  yet  been 
finalized  or  approved.  The  ILSP,  SMP,  and  the  CRMP  were  not  completed  or  signed. 
A  detailed  OSF  configuration  management  plan  needs  to  be  developed.  The  planned 
manning  levels  of  the  OSF  appeared  to  be  inadequate  to  perform  contract  monitoring 
functions  or  to  train  new-hires  and  support-contract  personnel.  Automated  support  tools 
for  software  development  and  configuration  management  were  insufficiently  addressed  in 
the  planning  documents  to  define  the  level  of  resources  required. 

5.17.2  Recommendations. 

5.17.2.1  For  JSPO: 

a.  Ensure  sufficient  automated  support  tools  are  available  to  support  configuration 
management,  quality  assurance,  and  software  development,  test,  and  distribution.  (SR  487) 

b.  Develop  an  adequate  configuration  management  plan  for  the  OSF. 

5.17.2.2  For  JSPO  and  users:  Ensure  the  ILSP,  SMP,  and  CRMP  are  coordinated  and 
approved. 

5.17.2.3  For  OSF: 

a.  Ensure  sufficient  resources  are  available  to  monitor  the  OSF  software  support 
contractor. 

b.  Develop  an  OJT  and  formal  follow-on  training  program  for  training  new-hires  and 
support-contract  personnel  after  the  one-time,  14-week  contractor- provided  training  course. 
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c.  Ensure  the  OSF  has  the  personnel  and  other  resources  necessary  to  maintain 
the  NEXRAD  software. 

5.18  OBJECTIVE  S-18.  Assess  NEXRAD  software  usability.  (Reference  paragraph  3.18) 

5.18.1  Conclusions. 

5.18.1.1  The  averages  for  the  operations  SUQ  indicated  generally  favorable  usability 
characteristics  for  the  PUP  and  UCP  software  interfaces.  Operators  stated  that  the  menu- 
driven  commands  enhanced  the  PUP  and  UCP  usability.  Operators  did  not  have  to 
memorize  commands  to  effectively  use  the  applications  terminals.  In  addition,  many 
product  manipulation  features  were  easily  invoked  using  the  graphics  tablet  (e.g.,  magnify, 
filter,  and  recenter).  However,  several  deficiencies  were  noted.  The  UCP  applications 
terminal  was  unable  to  accept  rapid  keyboard  inputs.  The  PUP  application  terminal  would 
not  execute  the  return  key  or  function  keys  when  the  screen  was  being  updated.  RCM 
editing  and  nonassociated  RPG  dialup  procedures  at  the  PUP  were  cumbersome.  The 
P'JP's  extended  adaptation  data  were  not  sufficiently  documented  and  required  extensive 
use  of  hexadecimal  codes.  Editing  procedures  for  the  UCP  edit  screens  were  inconsistent 
and  cumbersome.  Finally,  operators  had  difficulty  locating  information  in  the  PUP  and  UCP 
user’s  manuals. 

5.18.1.2  The  averages  for  the  maintenance  SUQ  indicated  generally  unfavorable  usability 
characteristics  for  the  evaluated  interfaces.  For  similar  functions,  the  MCC  and  UCP  menu 
structures  and  commands  were  unnecessarily  different.  The  maintainers  were  often 
required  to  copy  important  information  by  hand  because  printers  were  not  available  to 
support  the  maintenance  software  interfaces.  Finally,  the  technical  documentation  for  the 
diagnostic  interfaces  did  not  adequately  describe  needed  procedures  or  error  messages. 

5.18.2  Recommendations. 


5.18.2.1  For  JSPO: 

a.  Ensure  adequate  PUP  and  UCP  user’s  manuals  are  provided  with  an  index.  (SR 

403) 


b.  Include  adequate  menu  editing  orocedures  in  the  PUP  and  UCP  users'  manuals. 
(SRs  544.  543.  226,  333,  040,  175) 

c.  Provide  adequate  technical  documentation  for  the  Ramtek  diagnoses,  the 
RDASOT,  and  the  MCC  to  include  meaning  and  impact  of  error  and  status  messages. 
(SRs  129,  384.  355,  048,  066,  117,  151,  479,  388) 

d.  Eliminate  any  use  of  hexadecimal  code  for  character  or  numeric  input.  (SRs  41 5. 

438) 

e.  Eliminate  inconsistencies  in  UCP  and  PUP  editing  screens.  (SRs  029,  250,  309. 
015.  175,  161,  039) 

f.  Provide  compatible  interfaces  for  the  MCC,  UCP,  and  PUP.  (SRs  057.  164.  143) 

g.  Enable  applications  terminals  to  accept  keyboard  entries  during  screen  updates 
(SRs  167.  155) 

h.  Improve  RCM  editing  procedures.  (SRs  358,  333,  336,  428) 


V-13 


i.  Provide  effective  multiple  RPG  dialup  procedures  from  the  PUP.  (SRs  395,  393, 
394,  415,  515) 

5.18.2.2  For  JSPO  and  users:  Provide  a  print  capability  at  the  RDA,  RPG,  and  PUP  to 
support  the  Ramtek,  RDASOT,  and  the  Concurrent  Computer  Corporation  diagnostics  and 
the  MCC  interface.  (SRs  181  A,  082,  069,  125,  469) 

5.19  OVERALL  PERFORMANCE. 

5.19.1  Conclusions. 

5.19.1.1  When  the  overall  performance  of  N  EX  RAD  was  considered,  the  median 
questionnaire  response  of  ail  the  operators  indicated  that  the  system  did  not  meet  their 
requirements  as  an  aid  for  preparing  weather  warnings,  weather  advisories,  and  routine 
weather  services  (see  page  A-2).  Most  operators  stated  that  NEXRAD  was  often  not 
available  to  support  these  services  because  of  PUP  lockups,  system  outages,  and  problems 
with  recovering  automatically  from  power  transitions.  However,  possibly  because  of  their 
smaller  area  of  weather  support  responsibilities,  DOD  median  questionnaire  responses 
indicated  that  the  system  met  their  minimum  operational  needs  when  the  overall  NEXRAD 
performance  was  considered. 

5.19.1.2  When  the  operators  considered  the  overall  responsiveness  of  the  system  in  a 
multiple  user  environment  the  median  questionnaire  response  of  the  operators  indicated 
that  the  system  met  their  minimum  operational  needs  (see  page  A-2).  However,  possibly 
because  of  their  larger  weather  support  areas,  DOC  and  DOT  median  questionnaire 
responses  indicated  that  the  system  did  not  meet  their  minimum  operational  needs  when 
the  overall  NEXRAD  responsiveness  was  considered. 

5.20  FOLLOW-ON  OPERATIONAL  TEST  AND  EVALUATION.  To  refine  estimates  of 
operational  effectiveness  and  suitability  and  to  evaluate  changes  and  modifications  made 
to  correct  deficiencies  identified  by  the  IOT&E(2)  test  team,  the  using  agencies  should 
address  the  following  areas  during  FOT&E.  FOT&E  should  be  performed  in  an  operational, 
multiuser  environment  that  includes  associated  and  nonassociated  PUPs  from  all  using 
agencies  during  a  significant  weather  season.  The  system  should  include  limited  and/or 
full-scale  production  phase  capabilities  of  the  hydrology  algorithms,  full  RPGOP 
communications  speed,  and  the  limited  and  full-scale  production  phase  algorithms. 
Organizational-level  maintenance  should  be  performed  on  a  production-model  NEXRAD 
using  validated  and  verified  technical  manuals,  the  integrated  logistics  support  infrastructure, 
and  representative  training.  The  OSF  should  generate  and  test  a  new  software  version 
release  well  in  advance  of  the  SMRT  to  include  adding,  deleting,  and  changing  functionality 
within  the  RDA,  RPG,  and  PUP. 
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APPENDIX  A  -  MATRIX  OF  OPERATIONAL  TEST  RESULTS 


EVALUATED 


EFFECTIVENESS  OBJECTIVES 


Objective 

"WHEN  OPERATING"* 

All  Operators 

DOC 

DOD 

DOT 

Weather  Warnings  (E-1) 

M(4) 

M(4) 

M(5) 

- 

Operator  Workload  (E-2) 

PUP  Only 

M(2) 

M(2) 

M(2) 

D(1 ) 

PUP  and  UCP 

D(1 ) 

D(1 ) 

0(1) 

Multiple  Users  (E-4) 

M(4) 

M(4) 

M(5) 

D(3) 

Weather  Advisories  (E-5) 

M(4) 

M(4) 

M(5) 

M(4) 

Routine  Services  (E-6) 

Short  Range  Forecasts 

M(4) 

M(4) 

M(5) 

M(4) 

DOD  Surface  Observations 

M(4) 

— 

M(4) 

— 

Weather  Briefings 

M(4.5) 

M(4) 

M(5) 

M(4) 

DOT  Traffic  Management 

Briefings 

M(4) 

-- 

- 

M(4) 

M  =  Met  requirement 
D  =  Did  not  meet  requirement 
--  =  Not  applicable 


NOTE:  Numbers  in  parentheses  are  the  median  operator  questionnaire  response  ratings. 
For  objectives  E-1,  E-4,  E-5,  and  E-6  the  criterion  was  a  rating  of  4  or  greater  on  a  6- 
point  scale  (ranging  from  1  =  completely  ineffective  to  6  =  completely  effective).  See  table 
11-2  for  response  scale.  For  objective  E-2,  the  criterion  was  a  median  rating  of  2  or 
greater  on  a  5-point  scale.  See  table  11-3  for  response  scale. 


‘Operators’  responses  for  these  evaluations  were  based  on  only  when  the  system  was 
operating. 
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EVALUATED 


EFFECTIVENESS  OBJECTIVES 
"OVERALL  PERFORMANCE"* 


Objective 

All  Ooerators 

DOC 

DOD 

DOT 

Weather  Warnings  (E-1) 

D(3) 

0(2) 

M(4) 

- 

Operator  Workload  (E-2) 

PUP  Only 

M(2) 

M(2) 

M(2) 

0(1 .5) 

PUP  and  UCP 

D(1) 

0(1) 

0(1) 

- 

Multiple  Users  (E-4) 

M(4) 

0(3) 

M(4) 

D(1) 

Weather  Advisories  (E-5) 

0(3) 

0(2) 

M(4) 

D(2) 

Routine  Services  (E-6) 

Short  Range  Forecasts 

D(3) 

0(2) 

M(4) 

D(2) 

DOD  Surface  Observations 

D(3) 

— 

0(3) 

— 

Weather  Briefings 

D<3) 

0(2) 

M(4) 

D(2) 

DOT  Traffic  Management 

Briefings 

D(1) 

-- 

-- 

D(1) 

M  =  Met  requirement 
D  =  Did  not  meet  requirement 
--  =  Not  applicable 

NOTE:  Numbers  in  parentheses  are  the  median  operator  questionnaire  response  ratings. 
For  objectives  E-1,  E-4,  E-5,  and  E-6  the  criterion  was  a  rating  of  4  or  greater  on  a  6- 
point  scale  (ranging  from  1  =  completely  ineffective  to  6  =  completely  effective).  See  table 
11-2  for  response  scale.  For  objective  E-2,  the  criterion  was  a  median  rating  of  2  or 
greater  on  a  5-point  scale.  See  table  11-3  for  response  scale. 


‘Operators’  responses  for  these  evaluations  were  based  on  overall  performance  including 
impacts  of  PUP  lockups,  system  outages,  and  problems  with  automatically  recovering  from 
power  transitions. 
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EVALUATED 


SUITABILITY  OBJECTIVES 


Objective 
Maintainability  (S-13) 

MTTR 

Isolated  to  a  single  LRU 
Isolated  to  3  or  fewer  LRUs 

Availability  (S-14) 

Full  System 
Degraded  System 

Software  Maintainability  (S-16) 

Documentation 
Source  Listings 

M  =  Met  requirement 
D  =  Did  not  meet  requirement 


Results 


D 

D 

D 


D 

D 


M 

M 
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APPENDIX  B  -  PRIORITIZED  LIST  OF  CATEGORY  I  SERVICE  REPORTS 


List  of  Prioritized  Category  I  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 


Rank 

SR  # 

1 

168 

2 

012 

3 

010 

4 

264A 

5 

262A 

6 

190 

7 

189 

8 

Oil 

9 

009 

10 

076 

11 

049 

12 

061 A 

Title 

Safety  -  Unsafe  Power  Down  Procedures  for  Component  Replacement 
Safety  -  Hazards  Associated  with  the  Ufa  of  Radome  Davit  Assembly 
Safety  -  Hazards  with  Large  Radome  Hatch  Cover 
Personnel  Hazard  Due  to  Potentially  Unprotected  Hatch  Opening 
Personnel  Safety  Hazard  When  Opening  the  RDA  or  RPG  Tower  Room 
Floor  Hatch 

Safety  -  "Eye  Wash"  Required  in  RDA  Generator  Shelter 

Safety  -  Hazard  Associated  With  Exhaust  Fan  in  Generator  Shelter 

Safety  -  Inadequate  Safety  Railing  Around  Large  Radome  Hatch  Opening 

Safety  -  Hazard  Associated  with  Entry/Exit  Radome  Hatch  Opening 

Safety  -  Inadequate/Inappropriate  Fire  Suppression  Systems  at  IOT&E(2) 

Principal  User  Processor  (PUP)  Sites 

Safety  -  Generator  Shelter  Entrance  Hazard 

Unusable  Handrail  in  Tower 
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APPENDIX  C  -  PRIORITIZED  LIST  OF  CATEGORY  II  SERVICE  REPORTS 


List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 


ank 

SR# 

1 

098B 

2 

531 

3 

317 

4 

206 

5 

083 

6 

072B 

7 

006B 

8 

087B 

9 

014 

10 

265A 

11 

129 

12 

530 

13 

01 0A 

14 

396 

15 

168A 

16 

437 

17 

219 

18 

002B 

19 

440 

20 

164A 

21 

400 

22 

112A 

23 

138A 

24 

166 

25 

017 

26 

391 

27 

227 

28 

500 

29 

196 

30 

328 

31 

384 

32 

463 

33 

027B 

34 

138 

35 

286 

Title 

Potential  Transmitter  Reliability  Problem 
Too  Many  System  Status  False  Alarms 

Transfer  Between  Commercial  and  Back-Up  Power  Frequently  Forces 
The  RPG  into  an  Inoperable  Condition 

Spare  Transmitter  Line  Replaceable  Units’  (LRUs)  Configuration  Not 

Compatible  with  System  Under  Test 

Frequent  Ramtek  Graphics  Processor  Lock-Ups 

Undefined  RDA  Alarms 

Erroneous  Failure  Messages  on  System  Status  Menu 
Failure  of  Automatic  RPG  Restart 

Chapter  5  Preliminary  Technical  Manual  (PTM)  Inadequate 
Preliminary  Technical  Manual  Deficiencies 

Inadequate  Documentation  of  System  Status  Messages  at  the  RDA, 
UCP,  and  PUP  Applications  Terminals 

Deletion  of  and  Difficulty  in  Viewing  System  Status  Messages  at  UCP 
and  PUP 

Loss  of  Radar  Data 

New  Correction  Factors  for  Suncheck  Measurement  Subtest  1  (Align 
Pedestal)  Will  Not  Update  Correctly 
Audio  Alarms  at  the  UCP 

Numerous  PUP  Deficiencies  Apparently  Related  to  Graphic  Display  of 
Status  Messages 

Degraded  Operational  Utility  of  Base  Velocity  Products  Due  to  Range 
Folding 

Transmitter  Faults  Causing  Wedges  of  Missing  Data 

Numerous  Discrepancies  in  "RDA  Calibration"  Procedures 

Calibration  of  N  EX  RAD  Unit 

Corrupted  Links  In  Database  File 

Failed  Power  Transistor 

Undefined  PUP  System  Status  Messages 

System  Does  Not  Stay  On  Auxiliary  Power  When  Switchover  is 
Commanded  From  the  UCP  or  the  RDA  Maintenance  Terminal 
Orderly  Shutdown  of  RDA  at  RDA  Shelter  Not  Possible 
Safety  -  Inadequate  Waming/Caution  Signs  Throughout  NEXRAD 
NEXRAD  Transmitter  Field  Maintenance  Manual  (NWS  EHB  6-514) 
Inadequate 

Apparent  Velocity  Dealiasing  Errors 
Recenter/Magnify  Product  Function  Unreliable 
NEXRAD  Commercial  Manuals  of  The  Preliminary  Technical  Manual 
(PTM)  Inadequate 

Inadequate  Procedures  in  RDASOT  User’s  Guide  for  Generation  of 
Clutter  Map 

Safety  -  Personnel  Hazard  Associated  With  the  Fixed  Ladder  Attached 
to  the  Antenna 

Need  for  Audio  Alarm  for  Free  Text  Message  (FTM) 

Chapters  1-4  and  6  of  Preliminary  Technical  Manual  (PTM)  Inadequate 
Safety  -  Inadequate  Warnings  Located  in  Chapter  5,  Preliminary 
Technical  Manual  (PTM) 
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List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 


Rank 

SR  # 

Title 

36 

009A 

Need  for  Alert  of  System  Failure 

37 

420 

RDA  Transmitter  Beam  Voltage  Calibration  Data  Not  Available 

38 

207 

Safety  -  Inappropriate  Method  to  Bypass  Interlock  Switch  S4  in 
Transmitter  Cabinet 

39 

087 

Excessive  Acoustic  Noise  Associated  With  PUP  Cabinets 

40 

133 

Safety  -  Noncompliant  Grounding  and  Bonding 

41 

544 

Incomplete  Information  in  UCP  User’s  Manual 

42 

352 

Source  Listing  Preface  Block  Inadequacies 

43 

496 

Deficient  Sqfl^arqJJgcnqjgfltetion  Characteristics 

Safety  -  Inadequate/Unsafe  Radome  Obstruction  Light  Access 

44 

404 

45 

251 

RDA  System  Operability  Test  (RDASOT)  User’s  Guide  of  Preliminary 
Technical  Manual  (PTM)  Inadequate 

46 

543 

Incomplete  Information  in  PUP  User’s  Manual 

47 

178 

Inadequate  Free  Text  Message  (FTM)  Receipt  Notification 

48 

169 

Safety  -  Problems  Associated  With  Fire  Suppression  System  at  RDA 

49 

062B 

Apparent  Velocity  Aliasing 

50 

357 

Safety  -  Inadequate  Tower  Safety  Features 

51 

187 

Insufficient  Storage  Space  On-Site  for  Generator-  and  RDA-Related 
Support/Maintenance  Items 

52 

403 

Alphabetized  Indexes  for  PUP  User’s  Manual  and  UCP  User’s  Manual 
Not  Available 

53 

174A 

Unrealistic  Radials  of  Data  on  the  Base  Velocity  Product 

54 

354 

Inability  to  Center  Detected  RF  Pulse  During  RF  Pulse  Bracketing 
Alignment 

55 

368 

Archive  IV  Optical  Disk  Frequently  Unable  to  Support  "Archive  Write" 
Functions 

56 

239 

NEXRAD  Pedestal  System  Operation  and  Maintenance  Manual 
Inadequate 

57 

047B 

Inadequate  Plans  for  Proposed  RDA  Shelter  Interior 

58 

502 

Excessive  Narrowband  Loadshedding  of  Routine  Product  List  (RPS) 
Products  During  Widespread  Weather  Situations 

59 

506 

"Range-Folding"  Apparently  Caused  by  Clutter  Residue 

60 

169A 

Lack  of  Manual  RDA  Maintenance  Diagnostic  Procedures 

61 

497 

Deficient  Software  Source  Listing  Characteristics 

62 

323 

Inadequate  Data  Dictionaries 

63 

533 

Safety  -  Unsafe  Procedures  for  Access  to  Components  Located  at  the 
Top  of  the  Pedestal 

64 

059 

Unable  to  Specify  a  Start  Time  of  Data  for  PUP  Training  Mode 

65 

271 A 

Undefined  Alphanumeric  Keys 

66 

160 

Layered  Composite  Turbulence  Maximum  (LTM)  Values  Do  Not 
Correspond  to  Observed  Turbulence  Reports 

67 

061 

Optical  Disks  Jam  in  Optical  Disk  Drive  Units 

68 

050 

"CITY"  and  "COUNTY  NAMES"  Maps  Not  Available 

69 

032 

Safety  -  Easily  Accessible  Main  Power  On/Off  Switches  Required 

70 

098 

Safety  -  No  High  Voltage  Warning  Signs  on  Exterior  of  Transmitter 
Cabinet  or  on  Internal  High  Voltage  Points 

71 

113 

Safety  -  Telephone/Intercom  Communications  in  Radome  Area  Not 
Available 

72 

el  39  A 

Communication  Status  Messages 
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Rank 

73 

74 

75 

76 

77 

78 

79 

80 
81 

82 

83 

84 

85 

86 

87 

88 

89 

90 

91 

92 

93 

94 

95 

96 

97 

98 

99 

100 

101 

102 

103 

104 

105 


List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 

SR_#  Title 

301  Graphics  Processor  Locks-up  When  Attempting  to  Display  Products 
With  Apparently  Erroneous  Data 

532  Safety  -  Captive  Fasteners  Not  Used  Where  Required  by 

MIL-STD-1472C 

335  Descriptions  of  Complicated,  Mathematical  Algorithms/Calculations  Not 
Available  in  B5  and  C5  Documentation 

295  Functionality  for  Modifying  Current  Volume  Coverage  Pattern  (VCP) 
Not  Available 

058A  Excessive  Noise  Level  in  Operations  Room 

285  Safety  -  Power  Down  Requirements  and  Procedures  for  Preventive 
Maintenance  Inspections  Not  Specified 

383  Failure  of  PUP  Applications  Software  to  Verify  Results  of 

Operator-Initiated  Commands 

264  RDA  System  Operability  Test  (RDASOT)  Will  Not  Execute  After  RDA 
Applications  Software  is  Reloaded 

228  Rings  of  Missing  Data  on  Base  Products  for  2.4  Degrees  Elevation 
Slice 

159  Disconnected  Cabies/Wires  in  RDA  Cabinets 

096  Inadequate  Design  of  RDA  Shelter  to  Meet  Maintenance  Requirements 

416  Procedures  to  Use  RPG  Local  Maintenance  Terminal  Not  Available 

104A  Incorrect  NEXRAD  Status  Displays 

135  RPG  Maintenance  Console  Not  Fully  Functional 

055  Safety  -  Inadequate  PUP  Cable  Routing 

208A  Inadequate  Labeling  of  Line  Replaceable  Units  (LRUs) 

018  Programmable  Signal  Processor  (PSP)  Download  Error  When 
Attempting  to  Run  RDASOT  Diagnostics 
439  Description  of  System  Console  Mnemonic  Codes  and  Messages  Not 
Available 

077  Inadequate  List  of  UCP  System  Console  Commands  in  Preliminary 
Technical  Manual  (PTM) 

052  Safety  -  Insufficient  Clearance  to  Shut  Concurrent  3212  Cabinet  Door 
With  Control  Panel  Key  in  Place 

279  Problems  Associated  With  The  Display  of  Products  Recorded  on 
Archive  III 

297  Frequent  Degradation  of  RDA  Maintenance  Terminal  Operations 
088B  Automatic  RPG-PUP  Communications  Line  Connection  Failure 
441  Degradation  of  VAD  Winds  and  Velocity  Dealiasing  Algorithm 
108A  Inconsistent  Velocity  Patterns 

1 1 1 A  System  Dependence  on  Environmental  Control  Equipment  (Air 
Conditioning) 

399  Inadequately  Documented  "FILE  ADDRESS  ERROR"  Messages 
Received  During  Time  Lapse  Operations 
288  Inadequate  Documentation  for  Optical  Disk  Recovery  Procedures 
332  Potential  Reliability  Problem  Associated  With  5  1/4”  Optical  Disc  Drive 
Power  Switch 

181 A  Printer  for  UCP  Alphanumeric  (A/N)  terminals 
212  Insufficient  Information  on  Performance  Data  in  RDA  User’s  Guide 
208  Inadequate  Output  Available  from  Mesocyclone  Product 

026  Pulse  Forming  Network  (PFN)  Schematic  Not  Available  in  Preliminary 
Technical  Manual  (PTM) 
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Rank 

106 

107 

108 

109 

110 

111 

112 

113 

114 

115 

116 

117 

118 

119 

120 

121 

122 

123 

124 

125 

126 

127 

128 

129 

130 

131 

132 

133 

134 

135 

136 

137 

138 


List  of  Prioritized  Category  il  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(lA)  and  IOT&E(lB) 

SR#  Title 


382  Safety  -  Emergency  Lighting  and  Exit  Sign  Not  Available  in  RDA 
Shelter 

310  Safety  -  Problems  Associated  with  PUP  Workstation  Power  Strip 
(UD45A3) 

149  Suspected  Reliability  Problem  With  the  Pulse  Forming  Network  (PFN) 
Switch 

003  Undocumented  Operation  of  the  Auto  Display  Mode 
167  Intermittent  Response  to  <RETURN>  and  Function  Keys  When 
Entering  Commands  at  the  UCP  and  PUP  Applications  Terminals 
490  Apparent  Message  Framing  Inconsistency  Between  PUP  and  RPG 
and  Inadequate  Technical  Data  Support 
245A  Noncompliant  Grounding  and  Bonding 

329  Safety  -  Uncovered  Voltage  Terminal  Block  in  RDA  Data  Processor 
(UD5)  Cabinet 

120  Capability  to  List  the  Directory  of  the  Data  Stored  on  an  Archive  III  or 
Archive  IV  Optical  Disk  Not  Available 

380  Apparent  Inappropriate  Indications  of  Hail  from  the  HAIL  Product 
387  Safety  -  Pedestal  Platform  Bolts  Not  Safely  Accessible 
059A  Slow  Response  on  Alphanumeric  Terminals 

395  Insufficient  Capability  to  Dial  Non-Associated  RPGs 
197  Inadequate  Documentation  on  RDA  Data  Processor  (RDADP)  Power 
Supply  5/6  (PS5/PS6)  and  Power  Supply  7  (PS7) 

435  Procedures  to  Transfer  UCP  Functionality  from  UCP  to  RPG  Local 
Maintenance  Terminal  Not  Available 
517  Inability  to  Read  Status  Files  Archived  at  Separate  Times 
315  Safety  -  Improper  Location  of  the  RDA  Halon  Manual  Discharge  and 
Abort  Stations 

093  Safety  -  RDA  Data  Processor  (RDADP)  Power  Supply  Test  Points  to 
Close  to  Measure  Safely 

202  Test  Equipment  and  Tools  Not  Available  to  Perform  Hitachi  Adjustment 
and  Test 

413  RDA  Applications  Software  Did  Not  Run  When  the  Antenna  Exceeded 
the  Elevation  Electrical  Limits 

186A  Inadequate  Training  and  Documentation  on  the  Causes  of  Load 
Shedding 

124  Inadequate  Labeling/Marking  of  NEXRAD  Equipment 

525  Inadequate  Base  Velocity  Display 

130A  Missing  Base  Data  Using  Scan  Strategy  11 

084B  Need  for  Selected  Print  Capability  for  System  Status  File 

224  Archive  III  Products  Occasionally  Contain  Inappropriate  Data  Levels 

167A  Switch  Setting  Documentation 

096A  Suspected  Erroneous  Layer  Composite  Turbulence  Products 
201  Inadequate  Technical  Data  for  RDA  Data  Processor  (RDADP)  Power 
Supplies  (PS) 

355  Inadequate  Procedures  and  Functionality  to  Update  RDA  Adaptation 
Values 

393  Inadequate,  Time-Consuming  Procedures  for  Acquiring  Background 

Maps  from  Non-Associated  RPGs 

243A  Accessibility  Problem  with  PUP  Processor/Communications  Cabinet 

125A  Incorrect  Fuse  Labeling 
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List  of  Prioritized  Category  II  Service  Reports 

• 

Opened  during  IOT&E(2)  or  Revalidated  from  I0T&E(1A)  and  I0T&E(1B)  ] 

Rank 

SR  # 

Title 

139 

422 

Automatic  Pulse  Repetition  Frequency  (Auto  PRF)  Modification 

Functionality  Apparently  Not  Working 

140 

480 

PUP  Card  Strapping  and  Switch  Settings  Did  Not  Match  the 

141 

401 

Preliminary  Technical  Manual  (PTM) 

Inadequate  Documentation  of  RPG  Product  Limitations 

142 

482 

Klystron  and  System  Power  Indications  During  Warm-up  Not 

143 

446 

Documented 

Inadequate  Period  of  Retention  of  Free  Text  Message  (FTM)  Products 

144 

268 

Inadequate  RDA  Cable  Assemblies 

145 

447 

Inadequate  Functionality  of  PUP  Free  Text  Message  (FTM)  Product 

146 

402 

Edited  Clutter  Suppression  Region  Adaptation  Data  Reverting  to  Zeroes 

Following  "RPGUP" 

147 

130 

Inadequate  Notification  of  RDA  Request  for  Local  Control 

148 

173 

Verification  Prompts  Required  for  UCP  Commands  Which  May  Have 
a  Significant  Impact  on  NEXRAD  Operations 

149 

542 

Problems  Associated  with  PUPUP  and  RESTART  Command 

150 

493 

Index  for  CPCI  C5  Documentation  Not  Available 

151 

176 

Frequent,  Unrequested  Cleared  Screens  When  Displaying  Products  in 

Quarter  Screen  Mode 

152 

324 

Inconsistent  and  Inappropriate  Amounts  of  Data  Retrieved  From 

Archive  IV 

• 

153 

516 

Functionality  to  Read  and  Display  Archived  Status  File  From  Archive 

IV  Inoperable 

154 

538 

Safety  -  Water  in  RDA  Cable  Vault 

155 

082B 

Unterminated  Input  at  UCP  System  Console  Halts  RPG 

156 

095A 

Unavailable  Geographic  Annotations 

157 

499 

Possible  Base  Data  Positional  Inaccuracy 

158 

394 

Inadequate  Procedures  for  Requesting  More  Than  One  Product  When 

Dialing  a  Non- Associated  RPG 

159 

294 

Base  Reflectivity  Product  Occasionally  Contains  A r  nuthal  Appendages 
of  Reflectivity 

160 

511 

Initialized  Optical  Disk  Unusable  After  Reinitialization 

161 

325 

Archive  IV  Problems  when  Archiving  Entire  Product  Database 

162 

282A 

Halt  of  Product  Reception  when  "RPG  ERROR"  Messages  are 

Received 

163 

271 

Inability  to  Ascertain  Data  Level  of  a  Particular  Image  Display  Element 
in  a  Timely  Manner 

164 

457 

Auto  Archive  III  Did  Not  Automatically  Restart  After  an  RPG  Restart 

165 

062A 

Lost  System  Status  log 

166 

415 

Inadequate  Procedure  to  Update  RPG  Directory  (Extended  Adaptation 

Parameter,  Category  11) 

167 

228A 

Numerous  False  Alerts  from  Mesocyclone  Detection  Algorithm 

- 

168 

410 

Inadequate  Storage  Time  for  Background  Maps  from  Non-Associated 

RPG 

169 

248 

RDA  Performance  and  Maintenance  Data  Information  Not  Updated 

Before  Being  Displayed 

170 

231 

Safety  -  Receiver  Cabinet  Hazard  Caused  by  Improper  Mounting  of 

RF  Frequency  Generator 

• 

171 

005 

Concurrent  Functional  Schematic  35-770d08,  CPU-D  Not  Available 
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Rank 

172 

173 

174 

175 

176 

177 

178 

179 

180 

181 

182 

183 

184 

185 

186 

187 

188 

189 

190 

191 

192 

193 

194 

195 

196 

197 

198 

199 

200 
201 

202 

203 


List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 

SR#  Title 


414  Dissimilar  Data  Element  Names  in  C5  Documentation  and  Source 
Listings 

250  A  Undesired  Automatic  Scan  Mode  Change 

1 1 2  Documentation  on  Antenna  Pedestal  Level  Specifications  Not  Available 
537  Safety  -  Insufficient  "ON"  Period  for  RDA  Halon  System  Discharge 
Warning  Indicator 

381  Inadequate  Clearance  Between  Cabinet  and  Facility  Wall  to  Allow 
Access  to  Ramtek  Circuit  Cards 

056  Safety  -  Inadequate  Fastening  Method  for  RDA  Receiver  Cabinet 
Electromagnetic  Interference  (EMI)  Filter 
200  inadequate  Documentation  of  Status  Messages  Which  Refer  the 
Operator  to  a  Software  Technician  in  PUP/RPGOP  User’s  Manual 
036  Inadequate  Marking  of  Voltage  Terminals  in  the  Transmitter 
004  Undocumented  Error  Code  and  Recovery  Procedure  for  Keyboard 
Lockup  at  PUP  Applications  Terminal 
061 B  Anomalies  in  One-time  Requests 

515  Inability  to  Quickly  Identify  and  Enter  the  RDA/RPG  Mnemonic  for 
Dialing  Up  an  Nonassociated  PUP 

346  Front  Door  to  RDA  Data  Processor  (RDADP)  Cabinet  (Left  Bay)  Not 
Available 

265  Setup  Procedures  for  Micro  Junior  Fire  Control  Panel  Circuit  Board 
(UD1A5A1)  Not  Available 

484  Inconsistent  PUP  and  RPG  "Time  to  Begin  Edit"  and  Time  to  Edit" 
Checks  for  Radar  Coded  Message  (RCM)  Products 
008  "CONTINUE  ON  ERROR"  Option  Does  Not  Work  in  RDA  System 
Operability  Test  (RDASOT) 

290  CPCI-01  Documentation  Problems 

e073A  Uniform  Software  Commands 

405  Safety  -  No  Fire  Protection  Equipment  Located  in  Radome  Area 
132  Unorganized  Cable  and  Wire  Routing  of  Receiver  Cabinet  and  RDA 
Data  Processor  (RDADP)  Cabinets 
378  Inadequate  Documentation  for  Special  Test  Programs 

101  Preliminary  Technical  Manual  (PTM),  Table  5-2.11  Not  Available 
170  Software  and  Procedures  Required  to  Run  RDA  Special  Test  Programs 
Not  Available 

289  CPCI-04  Documentation  Problems 

360  Erroneous  Pick-A-Product  Operation 

522  Undocumented  Radial  Overlap  on  Base  Products 

407  Inconsistencies  Between  Communications  Interface  User's  Guide  for 
Class  III  and  Class  V  Users  and  Redbook 
053  Safety  -  AC  Outlet  Not  Available  in  Left  Rack  of  RDA  Data  Processor 
(RDADP) 

421  Ring  of  Missing  Data  on  LAYER  COMPOSITE  TURBULENCE 
MAXIMUM  -  MIDDLE  LAYER  (LTM)  Product 
412  Inaccurate/Incomplete  Documentation  of  Attenuator  Pads  and  Values 
535  AGC  (Automatic  Gain  Control)  Alignment  Procedures  Were  Time 
Consuming  and  Required  Special  Test  Fixture 
241  Current  Operational  Status  of  Archive  III  Not  Available 
492  Inadequate  Table  of  Contents  for  Each  CPCI  C5  Documentation 
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List  of  Prioritized  Category  II  Service  Reports 

• 

Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B)  1 

Rank 

SR  # 

Title 

204 

337 

Failure  of  UCP  Command  to  Connect  Communication  Line  for  an 

Individual  Associated  PUP 

• 

205 

258A 

Cumbersome  and  Time  Consuming  Radar  Coded  Message  (RCM) 

Editing  Procedures 

206 

351 

Archive  IV  "Append/Read  By  Time  Span"  Functions  Unreliable 

. 

207 

078 

Some  Transmitter  Voltage  Standing  Wave  Ratio  (VSWP)  Fault 

Adjustment  Equipment  Not  Available 

208 

453 

Inadequate  Applications  Training  on  Adaptable  Parameter  Changes 

209 

458 

Inadequate  AIRWAY  LOW,  AIRWAY  HIGH,  and  NAVAID  Background 

Maps 

210 

256 

Insufficient  PDA  Concurrent  3212  Internal  Cabling  Data 

211 

089A 

Product  Inaccuracies  Produced  by  Archive  Level  III  Record  and 

Playback 

212 

077B 

Lubricants  Leaking  at  Antenna  Pedestal  Assembly 

213 

e344 

Need  for  PUP  User’s  Manual  to  Contain  PUP  Power  On/Off 

Procedures 

214 

262 

Inconsistent  Part  Numbers  on  Line  Replaceable  Units  (LRU)  and  Part 

Numbers  Listed  in  Preliminary  Technical  Manual  (PTM) 

215 

476 

Safety  -  Diesel  Generator  (UD10)  has  Fuel,  Oil,  and  Exhaust  Leaks 

216 

390 

Error  When  Attempting  to  Display  Products  Read  From  Archive  IV 

217 

028 

Safety  -  Inadequate  Guards  and  Warning  Signs  In  Generator  Shelter 

218 

023 

Concurrent  3212  M80  Extender  Board  Not  Available 

• 

219 

214 

Safety  -  Hazardous  Exterior  Air  Vents  at  Generator  Shelter 

220 

081 B 

Expanded-view  Diagrams  Not  Included  in  Preliminary  Technical  Manual 
(PTM) 

221 

048 

Undocumented  RDASOT  Subtest  Error  Messages 

222 

033B 

Need  for  Self-retaining  Inserts  for  Mounting  Klystron  Air  Flow  Sensor 

223 

429 

Inadequate  PUP  Status  on  NEXRAD  Unit  Status  Graphic  Display 

224 

374 

CPCI-30  Documentation  Problems 

225 

001 

Safety  -  Sharp  Corners  on  Audio  Alarm  Case 

226 

300 

Method  for  Viewing  Hidden  RDA  Static  and  Occurrence  Alarms  Not 

Available 

227 

481 

Maintainer-lnitiated  RDA  "System  Test"  Not  Available 

228 

312 

Appropriate  RDA  Waveguide  Port  Attenuation  Figures  Not  Available 

229 

445 

Inadequate  Method  to  Display  Multiple  Free  Text  Message  (FTM) 

Products 

230 

0218 

Need  for  Capability  to  Monitor  Remote  UCP  at  RPG  Location 

231 

340 

Inadequate  Description  of  Software  Status  Messages  in  CPCI-04  C5 

Documentation 

232 

097 

Undocumented  Procedures  for  Bidirectional  Coupler  Replacement  and 

Calibration 

233 

267 

Capacitor  Analyzer  Not  Available 

. 

234 

170A 

Inadequate  Contour  Algorithm 

235 

483 

Oscilloscope  With  Storage  Capability  Required  for  Transmitter 
Troubleshooting 

236 

083A 

Inconsistent  Vertically  Integrated  Liquid  (VIL)  and  Severe  Weather 

Probability  (SWP)  Values 

237 

127A 

Inconvenient  Location  of  Fiber  Optic  Transceiver 

• 

238 

509 

Inadequate  Indication  of  Space  Remaining  on  Archive  Optical  Disk 
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List  of  Prioritized  Category  il  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  10T&E(1A)  and  IOT&E(1B) 


Rank 

SR  # 

239 

353 

240 

185 

241 

145A 

242 

e221 

243 

090 

244 

027 

245 

019 

246 

327 

247 

004A 

248 

093A 

249 

291 

250 

363 

251 

259 

252 

459 

253 

539 

254 

073 

255 

508 

256 

094 

257 

254 

258 

e013 

259 

057 

260 

158 

261 

397 

262 

191 

263 

252 

264 

071 

265 

468 

266 

314 

267 

el  50 

268 

064 

269 

095 

270 

031 

Title 


Inaccurate  Documentation  for  Performing  Transmitter  RF  Pulse 
Bracketing  Alignment 

Inability  to  Properly  Connect  High  Voltage  Power  Supply  Test  Cable 

With  High-Voltage  Connector 

Difficulties  in  Synchronizing  NEXRAD  Unit  Clocks 

Backup  Diagnostic  Tapes  for  the  3212  and  3280  Concurrent  Computer 

Are  Not  Available 

RPG  Power  On  Sequence  Documentation  Inadequate 
Safety  -  Wideband  Communications  Equipment  Not  Properly  Mounted 
Function  Key  Templates  Not  Available  for  All  PUP  Applications 
Terminals  (Concurrent  6312) 

Initial  Transmitter  Test  Equipment  Control  Settings  Not  Documented 
Misplaced  Radials 

Missing  Names  of  Airports  and  Counties 

Inadequate  RDA  Generator  Fuel  Level  Indications 

Wrong  Index  Values  and  Missing  Checks  in  Module 

A403YE_RES_RQST_RESP 

Safety  -  Tripping  Hazard  Associated  with  Electrical  Conduit  in  Radome 
Occasionally  Unable  to  Cancel  Archive  Read 
Opening  Radome  Hatch  Did  Not  Cause  "RADOME  ACCESS  HATCH 
OPEN"  Alarm  at  RDA  and  UCP  Consoles 

Bidirectional  Coupler  Not  Calibrated  for  Total  NEXRAD  Frequency 
Range 

Product  Annotation/Status  Area  of  Graphic  Display  Difficult  to  Use 
Unable  to  Complete  Concurrent  3200  Series  Multiple  Peripheral 
Controller  (MPC)  Diagnostics  Test 

"<U>nit  Control,  <R>estart"  Command  Frequently  Does  Not 
Automatically  Restart  the  RPG 

Need  for  an  Easily  Executable  Method  to  Save  and  Load  Adaptation 
Data  Under  Operator  Configuration  Control 

Inoperative  "DISPLAY  PERFORMANCE  DATA"  (DIPD)  Command  ?t 
RDA  Maintenance  Control  Console  (MCC) 

Input  "ffer  (Wideband)  Loadshedding 

Archive  IV  "OPTICAL  DISK  FULL"  Message  Not  Received 

Inadequate  UCP  "COMMUNICATIONS  STATUS"  Menu  and  Support 

Documentation 

Preliminary  Technical  Manual  (PTM),  Revision  "C",  Chapter  5.  is 
Missing  Maintenance  Procedures 

Safety  -  Location  of  Archive  Device  Routinely  Exposes  PUP  Circuit 
Boards  to  Accidental  Damage 

Inadequate  Maintenance  Concept  for  Color  Graphics  Printer 

PUP  Cables  Being  Damaged  in  Concurrent  3212  Processor  Chassis 

Need  For  "NEXRAD  UNIT  STATUS"  Products  to  Include  Information 

on  Meteorological  Impact  of  Alarm  Conditions 

Inability  to  Perform  Concurrent  3212  Error  Logger  Test  #250,  Subtest 

4  Diagnostics 

Unable  to  Perform  Voltage  Standing  Wave  Ratio  (VSWR) 
Measurements 

Safety  -  Inadequate  Access  to  Voltage  Measurement  Points  in  RDA 
Data  Processor  (RDADP)  Cabinet 
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List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  I0T&E(1A)  and  IOT&E(1B) 


Rank 

SR  # 

271 

193 

272 

225 

273 

474 

274 

223 

275 

062 

276 

249 

277 

477 

278 

el  64 

279 

118 

280 

016 

281 

230 

282 

240 

283 

203 

284 

349 

285 

006 

286 

495 

287 

255 

288 

151 A 

289 

307 

290 

238A 

291 

308 

292 

419 

293 

103 

294 

022 

295 

487 

296 

491 

297 

066 

298 

367 

299 

073B 

300 

417 

301 

541 

302 

526 

303 

065B 

304 

042 

Title 


Incorrect  Narrowband  Communications  Line  Status  Displayed  Following 
a  Narrowband  Line  Disconnect 

Safety  -  No  Power  Switch  for  Radome  Heater  in  Radome 

Radio  Frequency  (RF)  and  Intermediate  Frequency  (IF)  Test  Monitor 

Component  Location  Impairs  Maintenance 

Attributes  Table  Not  Archived  With  Composite  Reflectivity  Product  on 
Archive  ill  and  IV 

Time  Lapse  "CONTINUOUS  LOOP"  Occasionally  Stops  Without 
Operator  Interaction 

Erroneous  Data  Occasionally  Appears  in  Lowest  Two  Elevation  Slices 
of  Base  Data  Products 

Illustrated  Parts  Breakdown  (IPB)  Not  Available 

Confusing  Commands  on  RDA  Control  Menu  at  UCP  and  Main  Menu 

at  RDA  Maintenance  Control  Console  (MCC) 

Safety  -  Removal  of  Transmitter  Modulator  Pulse  Assembly  Top  Cover 

May  Damage  Printed  Circuit  Board  UD3A12A8 

Concurrent  3212  Power  Supplies  (P5  and  P5U)  are  Too  Heavy  for 

Supports 

Inadequate  Color  Graphics  Printer  Maintenance  Documentation 
Initial  Archive  IV  Status  Information  Inappropriate  if  Optical  Disk  Not 
in  Optical  Disk  Drive 

PUP  Does  Not  Display  Latest  Available  Severe  Weather  Products 
Deficiencies  Associated  With  "RIVER  BASIN"  Map 
Inadvertent  Tripping  of  Concurrent  3212  Power  Supply  Circuit  Breaker 
(CB) 

Unique  Naming  Convention  For  Global  Data  Elements  Not  Used 
Halon  Gas  Pressure  Gauge  and  Motorized  Damper  Assembly  PMIs 
Procedures  Not  Available 

Loss  of  Operator-entered  Parameters  During  Reboot 

Incorrect  Centroid  Heights  in  Part  C  of  the  Radar  Coded  Message 

(RCM) 

Missing  Information  on  Rivers  Background  Maps 

Inconsistent  Height  Values  for  Winds  in  the  Radar  Coded  Message 

(RCM),  Part  B 

Unambiguous  Velocity/Range  Information  Not  Readily  Available  for 
Velocity  Products 

Unable  to  Perform  Transmitter  Oil  Dielectric  Strength  Test 

Inability  to  Exercise  Small  Computer  System  Interface  (SCSI)  Using 

Copy  Task  "COPY32"  as  Indicated  in  PTM 

Inadequate  System  Dump  Analysis  Procedures 

Inefficient  Format  for  CPC  Descriptions  in  the  C5  Documentation 

Undocumented  Concurrent  3212  Subtest  Error  Messages 

Archive  III  Loadshed  Warning  Message  Not  Displayed 

Inadequate  Loadshed  Category  Information 

RF  Pulse  Shaper  Waveform  Checks  Procedures  Not  Available 

Crash  Codes  Not  Documented 

Inadequate  and  Duplicative  Interface  Information  in  ICD  and 
Communications  Interface  User’s  Guide 
Inadequate  Weak  Echo  Region  (WER)  Product 
Intermixed  Archive  II  and  Archive  III  Functions  on  Same  Menu 
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List  o*  Prioritized  Category  il  Service  Reports 

Opened  during  IOT&E(2)  or  Revalidated  from  I0T&E(1A)  and  IOT&E(1B) 

• 

Rank 

SR  # 

Title 

305 

248A 

Inadequate  Training  on  Dial-up  Procedures 

306 

e246 

Need  for  Adequate  Method  to  Notify  Technician  at  RDA  Site  That 
Telephone  is  Ringing 

* 

307 

362 

CPCI-06  Documentation  Problems 

308 

e236A 

Need  to  Save,  Load,  and  Catalog  Adaptation  Data  Files 

309 

01  OB 

Inadequate  Quadrant  Display  of  One-time  Requested  Products 

310 

408 

Inadequate  "Elevation  Up"  and  "Elevation  Down"  Functionality 

311 

430 

Inadequate  Range  of  the  VIL  Product 

312 

117 

Inadequate  Ramtek  4660  Diagnostics  Test  10  Documentation 

313 

272 

Inadequate  Documentation  on  the  Effects  of  Powering  Down  Data 
Acquisition  Unit  (DAU) 

314 

088 

Inadequate  Cabinet  Door  Hinges 

315 

245 

Procedures  for  Use  of  RDA  3212  Processor  Multiple  Peripheral 
Controller  (MPC)  Diagnostics  Not  Available 

316 

151 

Inadequate  Ramtek  4660  Diagnostics  Test  14  Documentation 

317 

e255A 

Need  for  Display  of  RDA/RPG  Performance  and  Maintenance  Data 

318 

514 

Non-Receipt  of  Products  Via  Dial-up  When  WARNING  AREA  or  RDA 
Maps  are  Associated  with  Product 

319 

002 

Loss  of  Time  Lapse  Parameters  Following  a  PUP  Shutdown  and 
Restart 

320 

342 

CPCI-03  Documentation  Problems 

321 

081 

Terminal  Configuration  Data  Not  Available 

322 

162 

Failure  to  Display  Most  Current  Version  of  Selected  Products  When 
in  Training  Mode 

# 

323 

281 

Inadequate  "RIVER"  Map 

324 

296 

Safety  -  Problems  Associated  With  RDA  Sheiter  Door 

325 

080 

Safety  -  Data  Acquisition  Unit  (DAU)  Maintenance  Panel  Hazard 

326 

418 

Graphics  Lock-up  Apparently  Associated  with  Color  Printer  Paper  Jam 

327 

545 

Inability  to  Adequately  Specify  Required  Frequency  of  Archive  III 
Products 

328 

194 

Inability  to  Designate  One-Time-Requested  Products  for  Automatic 
Archiving  on  Archive  III 

329 

038 

Safety  -  Line  Replaceable  Units  (LRU)  In  Excess  of  One-Person  Lift 
Values  Not  Labeled 

330 

268A 

RPG  Can  Generate  Only  Two  Reflectivity  Cross-section  (RCS) 
Products  per  Volume  Scan 

331 

122 

Inadequate  Editing  Capability  for  High  Resolution  "CITY"  Map 

332 

193A 

Loss  of  Velocity  Azimuth  Display  (VAD)  Data  During  RPG  Reboot 

333 

338 

Archive  IV  Commands  to  Archive  and  Read  Received  Background 
Maps  for  a  Selected  RPG  Not  Available 

- 

334 

136 

Undocumented  Archive  Level  IV  Error  Messages 

335 

084 

Unable  to  Perform  Transmitter  Meters  A1  Ml .  A1  M2.  A1  M3  Adjustments 

336 

386 

Inappropriate  Alarms  on  the  Wideband  Fiber  Optics  Transceiver 

' 

337 

226 

Safety  -  Hazardous  Condition  Associated  with  Transmitter  Intake  Air 
Vent 

338 

094A 

Metric  Units 

ft 

339 

479 

Inadequate  Documentation  on  RDA  Maintenance  Terminal  Options 

340 

229 

Inadequate  Method  to  Remove  PUP  Ramtek  Graphics  Processor  Hard 
Cursor  Card  (UD41A13A4) 

341 

528 

Passwords  Used  Within  the  Functional  Areas  Not  Available 
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List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 


Rank 

SR  # 

342 

142 

343 

438 

344 

034B 

345 

518 

346 

442 

347 

092 

348 

292 

349 

448 

350 

347 

351 

361 

352 

540 

353 

304 

354 

153 

355 

472 

356 

144 

357 

478 

358 

139 

359 

501 

360 

155 

361 

432 

362 

134 

363 

209 

364 

e053B 

365 

326 

366 

033 

367 

199 

368 

192 

369 

049B 

370 

454 

371 

059B 

372 

343 

373 

253 

374 

498 

Title 


Inadequate  Documentation  to  Determine  Average  Pulse  Repetition 
Frequency  (PRF) 

Inadequate  User  Interface  for  Extended  Adaptation  Editing  Procedures 
Strain  Relief  Needed  on  Transmitter  Multiconductor  Cable 
Inability  to  Read  Background  Maps  From  Archive  IV 
Inadequate  Storm  Tracking  Algorithm 

Accessibility  Problem  with  Modulator  Pulse  Assembly  (UD3A12) 
Improper/Missing  Cable  Hardware 

Product  Forward/Back  Function  on  Alphanumeric  (A/N)  Terminal 
Unavailable 

Damaged  RDA  Data  Processor  (RDADP)  Cabinet  Cable  5A10J21 
Procedures  to  Inspect  and  Clean  Azimuth  Siiprings  and  Brushes  Not 
Available 

RDA  Maintenance  Terminal  Displayed  Garbled  Data  When  Using  Page 
Backward  Command  in  Receiver  Adaptation  Tables  Screen 
FMH  1 1  inadequate  Documentation  for  Storm  Attributes  Overlay 
Inadequate  Suspension  of  Waveguide  Switch  Assembly 
Inadequate  Crimp-on  Connectors  Used  on  PUP  Line  Filter  (UD41FL1) 
Power  Cable 

Inadequate  Routing  of  RDA  Receiver  Cabinet  (UD4)  Log  Video  Cable 
Access  to  AGC  Threshold  Control  Not  Visible  During  Maintenance 
Operation  of  RDA  Data  Processor  (RDADP)  Cabinet  (UD5) 
Maintenance  Panel  (A2)  Audible  Alarm  Not  Documented 
Inadequate  UCP  User’s  Manual  Documentation  for  Archive  III 
Automatic  Archiving  Procedures 

Degradation  of  Applications  Terminals’  Responsiveness  When  Frequent 
Alarm  Messages  are  Being  Displayed 

Ramtek  Graphics  Processor  External  Test  Points  Not  Available 
RDA  Data  Processor  (RDADP)  Cabinet  Doors  Will  Not  Close  Due  to 
Cable  Routing 

Inadequate  Support  Equipment  to  Perform  Security  Alarm  Installation 
and  Modification 

Need  for  Ability  to  Display  and  Archive  Algorithm-generated  Parameter 
Values  at  the  PUP 

Inability  to  Archive  Dial-up  Background  Maps  to  Archive  IV 

Safety  -  Inadequate  Fastening  Method  in  Transmitter  High  Voltage 

Cabinet 

Inappropriate  "PRESENT  TIME"  Function  at  PUP  Graphic  Tablet 
Proper  Tools  and  Support  Equipment  for  Transmitter  VSWR  Fault 
Circuit  Adjustment  Not  Available 

Need  for  Volume  Coverage  Pattern  (VCP)  Unambiguous  Velocity  and 
Range  Information 

Inappropriate  Response  to  ELEVATION  UP/DOWN  Function  Selection 
List  of  Dial-up  Products  Accessible  by  Nonassociated  PUPs/RPGOPs 
Not  Available 

Inconsistent  Documentation  of  Attenuation  Path  Used  in  RDA 
Calibration  Calculations 

Inadequate  Weak  Echo  Region  (WER)  Product 

Comprehensive  Glossary  for  CPCI  C5  Documentation  Not  Available 
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Rank 

375 

376 

377 

378 

379 

380 

381 

382 

383 

384 

385 

386 

387 

388 

389 

390 

391 


392 

393 

394 

395 

396 

397 

398 

399 

400 

401 

402 

403 

404 

405 

406 


List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E{2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 

SR  #  Title 

082  Insufficient  RDASOT  User’s  Guide  Procedures  for  Recording  and 
Printing  Maintenance  Session 

232  Unable  to  Locate  Transmitter  Test  Points  on  the  A1A2  Module 
507  Inadequate  Capability  to  Specify  Velocity  Products  Data  Levels  at  PUP 
186  Test  Points  for  Transmitter  Modules  Difficult  to  Access  When 
Transmitter  is  Operating 

141  Information  on  Time  Required  to  Complete  Diagnostic  Tests  Not 
Available 

217  Inability  to  Determine  the  Reliability  of  the  Velocity  Azimuth  Display 
(VAD)  Computed  Wind 

536  inability  to  Complete  PUP  Color  Monitor  Adjustments 
029  Inadequate  Environmental  Winds  Editing  Capabilities 
467  RDA  Display  Performance  (DIPD)  Menus  Incomplete  and  Not 

Adequately  Documented 

302  Inadequate  Documentation  to  Correlate  NEXRAD  Product  Formats 
With  Redbook  Transmission  Blocks 

311  Date  Variable  Improperly  Incremented  in  Module 

A3052H_P  R  EC  I  P_C  ATS 

278  Accessibility  Problem  With  the  Radar  Product  Generation  (RPG) 

Input/Output  (I/O)  Panel 

e085B  Need  for  Uniform  Free  Text  Message  Send/Display  Procedures 
067  Safety  -  Inadequate  Dummy  Load  Support 

388  Inappropriate  Messages  on  PUP  Applications  Terminal  While 

Performing  Ramtek  Diagnostics  Test  14  Subtest  1 
070  Procedures  for  Initializing  Opt12al  Disks  Not  Available 
126  Incorrect  PUP  User’s  Manual  Instructions  for  Loading  and  Unloading 
Archive  IV  Optical  Disks 

102  Beam  Voltage  Proximity  Sample  Line  Not  Connected  to  the  Transmitter 
Oil  Tank  Test  Jack  J1 

358  Automatic  Editing  of  Radar  Coded  Message  (RCM)  Parts  A  and  C  For 
No-Echo  Conditions  Not  Available 

266  Inadequate  Clearance  for  Removal  of  Air  Conditioner  Economizer 
Filters 

461  Inadequate  Linking  of  Cursors  in  Dual  Four  Quadrant  Graphic  Mode 
313  Inadequate  RDA  Shelter  Waveguide  Installation 
377  Inappropriate  Duplication  of  CPCI-24  Shared  Modules 

034  Safety  -  First  Aid  Kit  Not  Available  at  RDA  ter 

426  Maintenance  and  Replacement  Procedures  for  -  >eo  Horn  Assembly 
and  Waveguide  Sections  Forward  of  Reflecto;  >t  Available 
258  Security  Panel  Documentation  Not  Available 

068  Deficiencies  Associated  With  Transmitter  Meter  AIM4 
069  Inadequate  UCP  Applications  Terminal  Screen  Printout  Capability 
066B  Restrictions  on  User  Functions 

521  Apparently  Inappropriate  Error  Message  When  Selecting  Archive  IV 
Auto  Archiving  Frequency 

250  Edits  of  Current  Generation  and  Distribution  Control  List  Also  Changes 
Adaptation  List 

392  PUP  Workstation  Color  Monitors’  (UD45)  Control  Knobs  Not  Securely 
Mounted 
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List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(lB) 


Rank 

SR  # 

407 

el  28 

408 

273 

409 

527 

410 

462 

411 

226A 

412 

e021 

413 

460 

414 

243 

415 

035 

416 

210 

417 

237 

418 

070B 

419 

e293 

420 

140 

421 

148 

422 

099 

423 

e235A 

424 

065 

425 

451 

426 

320 

427 

072 

428 

471 

429 

371 

430 

054B 

431 

e064B 

432 

366 

433 

370 

434 

488 

435 

079 

436 

e220 

437 

el  54 

438 

eOlIB 

439 

e190A 

Title 


Need  for  Capability  to  Quickly  Identify  Changed  System  Parameter  on 
NEXRAD  Unit  Status  Display  Graphic 

Unsecured  Hardware  on  Pedestal  Power  Amplifier  Air  Filter  (UD5A7) 
Safety  -  Cover  For  Fluorescent  Lamp  in  Transmitter  Cabinet  Not 
Provided 

Generator  Shelter  Telephone/Intercom  Communications  Not  Available 
Duplicate  Use  of  Storm  Identification  Numbers 
Need  for  Metric  Units/English  Units  Toggle  Function 
AUTO  DISPLAY  Functionality  Not  Available  in  Training  Mode 
Incorrect  Coding  of  "Operational  Mode"  on  Radar  Coded  Message 
(RCM)  Alphanumeric  Product 

Safety  -  First  Aid  Kit  Not  Available  at  Generator  Shelter 

Inability  to  Modify  Storm  "TRACKING  AND  FORECAST"  Default  Speed 

Source  Code  Files  A317M3  and  A317M4  Not  Available 

Need  for  Test-point  Markings  on  Transmitter  Inner  Door 

Color  Graphic  Printer  Copy  Counter  Instructions  Not  Available 

Incomplete  Documentation  on  Receiver  Power  Supplies  Voltage 

Tolerances 

Inadequate  Length  of  RDA  Data  Processor  (RDADP)  Pedestal  Control 
Unit  <PCU)  Cable 

Reference  Designators  on  Waveguide  Components  Are  Not  Consistent 

With  Unisys  Technical  Data 

Need  for  Multiple  Color  tables  for  Each  Product 

Concurrent  3212  Design  Inappropriate  for  Running  MAT/CACHE  Test 

Diagnostics 

Inadequate  Retention  of  Operator-Specified  Storm  Motion  Parameter 
Values 

FMH-1 1  -  Hail  Product  Functional  Description  Does  Not  Document  Hail 
Size  That  Algorithm  Was  Designed  to  Detect 

Inadequate  Fastening  Method  for  Transmitter  (Tx)  Air  Intake  Filter 
Cover 

RDA  System  Operability  Test  (RDASOT)  Displayed  Incorrect 
Maintenance  Action 

Commercial  Manual  for  Diesel  Generator  Shelter  Exhaust  Fan  Not 
Available 

Need  to  Specify  Altitude  Level  on  the  Velocity  Azimuth  Display  (VAD) 
Alert 

Need  for  NEXRAD  Unit  Status  Graphics  Product  to  Display 
Automatically  Upon  VCP  Changes 

Inconsistent  Time  on  Alert-Paired  SEVERE  WEATHER  ANALYSIS  - 

REFLECTIVITY  (SWR)  Product 

Source  Code  Module  A4CM40  Not  Available 

CPC-17  Modules  Incorrectly  Located  with  CPC-18  Modules 

Procedures  for  Cleaning  and  Inspecting  Concurrent  6312  Terminals  Not 

Available 

Need  for  Improved  Monitoring  of  RDA  Area  by  Security  System 
Need  for  an  Easily  Executable  Method  to  Save  and  Load 
Unit- Radar-Committee  (URC)-Controiled  Adaptation  Data 
Need  for  Concurrent  Screen  Update  Capability 
Inconsistent  Use  of  Coordinate  Systems 
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List  of  Prioritized  Category  il  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 


Rank 

SR  # 

440 

411 

441 

431 

442 

123 

443 

104 

444 

e216A 

445 

e224A 

446 

180 

447 

165 

448 

373 

449 

184 

450 

333 

451 

221 A 

452 

045 

453 

504 

454 

e348 

455 

427 

456 

el  83  A 

457 

280 

458 

183 

459 

406 

460 

099A 

461 

356 

462 

494 

463 

el  43 

464 

303 

465 

519 

466 

172 

467 

e046B 

468 

215 

469 

283 

470 

e071B 

471 

336 

472 

e047 

473 

260 

474 

001 A 

475 

107 

476 

137 

Title 


Inadequate  Radar  Coded  Message  (RCM)  Edit  Audio  Alarm 
Missing  Slices  on  WEAK  ECHO  REGION  Product 
"PRECIPITATION  DETECTION"  Edit  Screen  Rate  Threshold  Umits 
Too  Low 

Transmitter  Oil  Pump  Motor  Lubrication  Holes  Difficult  to  Access 
Need  for  On-line  Maintenance  Logs 

Need  for  Product-specific  "CONTROL"  and  "PRODUCT"  Edit  Screens 

Inadequate  Default  Weather  Mode  Functionality 

Erratic  Voltage  Adjustment  in  the  RDA  Data  Processor  (RDADP) 

Procedures  to  Perform  the  PUP  Color  Monitor  Operational  Check  Not 

Available 

Inability  to  Set  Equalizer  Functions  on  Fujitsu  Modem 
Inadequate  Documentation  in  PUP  User’s  Manual  on  Radar  Coded 
Message  (RCM)  Editing  Procedures 

Velocity  Azimuth  Display  (VAD)  Product  Data  Not  Available  at  1000 
ft  Intervals 

RDA  Data  Processor  (RDADP)  Cabinet  (UD5)  Power  Supplies  (PS) 
Not  Properly  Labeled 

Severe  Weather  Probability  (SWP)  Color  Data  Levels  Not  Included  on 
Display 

Need  for  Capability  to  Retain  Previous  Graphic  Product  Manipulations 
for  Use  in  Subsequent  Product  Displays 
Cumbersome  Procedure  for  Transmitting  the  Edited  RCM 
Need  for  Echo  Tops  Contour  Overlay 

Inadequate  Detail  on  "HIGHWAY"  Map-Both  High  and  Low  Resolution 
Inability  to  Complete  Codex  2260  Modem  Retrain  Function  Test 
Corrosion  on  Unprotected  Area  of  Antenna  Pedestal  Torque  Tube 
Lack  of  Navaid  Legends 

Noninterchangeable  Alphanumeric  Terminals  and  Noninterchangeable 
Keyboards 

Incomplete  Version  Description  Documents  (VDDs) 

Need  for  Improved  RDA  Applications  Terminal  Menu  Interface 
Inconsistencies  in  Alert  Units  Between  "ALERT  THRESHOLD  VALUES" 
and  "ALERT  PROCESSING  EDIT  SCREEN  HELP  SCREEN" 
Inadequate  Real-Time  Simulation  During  Archive  Retrieval  in  Training 
Mode 

Inconsistent,  Non-Standard  Color  Coding  of  Receiver  (UD4)  Power 
Supply  1  (PS1)  Test  Points 
Need  for  Turbulence  Alert 

Inappropriate  Error  Message  Following  Modification  of  STF  Default 

Direction  Adaptation  Parameter 

Inadequate  Documentation  on  RDA  Pedestal  Bolts 

Need  for  Improved  Method  to  Page  Through  Help  Menus 

Inability  to  Delete  All  Centroids  From  the  Radar  Coded  Message 

(RCM)  Graphic  Product 

Need  for  Capability  to  Center  Polar  Grid  at  Any  Desired  Location 
Technical  Manual  for  RDA  Shelter  Thermostats  Not  Available 
Time  Lapse  Speed  Inconsistent 

Unable  to  Perform  Cabinet  Blower  B3  Belt  Tension  Adjustment 
Unattached  Panel  Stiffener/Cover  Spacers 
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List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  I0T&E(1B) 


RanK 

SR  4 

Title 

477 

163 

Color  Selection  Mode  Does  Not  Automatically  Cancel  When  Another 
Product  is  Displayed 

478 

e263 

Need  for  "REDISPLAY  LAST  PRODUCT"  Function  to  Retain  All 
Previous  Manipulations 

479 

309 

Frequent,  Unrequested  Cancellation  of  the  Graphic  Color  Selection 
Process  During  Left  Graphic  Screen  Editing 

430 

030 

Receiver  Power  Supply  (UD4PS1)  +VA  Adjustment  Not  Labeled 

481 

098A 

Unknown  Product  Ranges 

482 

503 

Difference  Between  UCP  User’s  Manual  and  PUP  User’s  Manual 
Concerning  RUES  Port  Background  Map  Distribution 

483 

040 

Inadequate  Documentation  on  Usage  of  Environmental  Winds  Edit 
Screen 

484 

01 2B 

Inconsistent  Archive  Menus  at  the  PUP  and  UCP 

485 

284 

Radome  Ventilation  Fan  Preventive  Maintenance  Procedures  Not 
Available 

486 

444 

PUP  User’s  Manual  "Cross  Reference  of  Command5Functions”  Not 
in  Alphabetical  Order 

487 

e331 

Need  for  Overlay  Product  Showing  Trend  of  Maximum  VIL  and  SWP 
for  Current  Storm  Centroids 

488 

257 

PMI  for  Inspecting/Cleaning  Pedestal  Power  Amplifier  (UD5A7)  Air  Filter 
Not  Available 

489 

el  25 

Need  for  Printer  at  the  RDA  Site 

490 

195 

Inconsistent  Operation  of  "CANCEL  USER  FUNCTION"  Command 

491 

529 

Incorrect  Documentation  of  Test  Equipment  Configuration  For 
Transmitter  Alignments 

492 

147 

Rear  Panel  of  Wideband  Fiberoptics  Transceivers  Will  Not  Close 

493 

204 

"RAMTEK  HARDWARE  HELP  SCREEN”  Contains  Inappropnate 
Procedures 

494 

449 

Numerous  Deficiencies  in  FMH-11,  Part  D.  Chapter  4  (DRAFT) 

495 

e071  A 

Transmitter  Voltage/Current  Meter  and  Selector  Switch 

496 

505 

Training  Mode  Status  Messages  Not  Available 

497 

466 

Safety  -  Inadequate  Mounting  of  AC  Power  Junction  Box 

498 

473 

Procedure  to  Verify  and  Adjust  the  Elevation  Pre-limit  and  Final  Limit 
Switches  Not  Available 

499 

e269 

Need  for  Improved  Contour  Functionality  at  The  PUP 

500 

244 

Incorrect  Switch  Orientation  on  RDA  Maintenance  Panel  (UD5A2) 

501 

277A 

Need  for  Reset  Function  for  Alerts 

502 

375 

Pedestal  Azimuth  Assembly  (UD2A1A3)  Oil  Sight  Glass  Leaking  Oil 

503 

127 

Automatic  Archiving  Does  Not  Restart  Following  a  PUP  Restart 

504 

e022B 

Need  for  Simplified  Color-table  Editing  Procedures 

505 

043 

Inconsistency  Between  UCP  Archive  Help  Screen  Information,  UCP 
Archive  Menu,  and  UCP  User’s  Guide 

506 

015 

Inconsistent  Threshold  Editing  Procedures  within  Alert  Processing  Edit 
Screen 

507 

534 

Selected  RDA  Command  Mnemonics  for  Installed  Equipment  and 
Function  Not  Executed 

508 

452 

CITY  Map  Names  Overlap  in  Quarter-Screen  Mode 

509 

e510 

Need  for  Improved  Point  Echo  Rejection  in  Echo  Tops  Algorithm 

510 

298 

Inadequate  Method  of  Attaching  Micro  Junior  Fire  Control  Panel 
(UD1A5)  Terminating  Resistors 
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Rank 

511 

512 

513 

514 

515 

516 

517 

518 

519 

520 

521 

522 

523 

524 

525 

526 

527 

528 

529 

530 

531 

532 

533 

534 

535 

536 

537 

538 

539 

540 

541 

542 

543 

544 

545 


List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 


SR  # 


Title 


238 

434 

117A 

6277 

261 

376 

319 

156 

489 

145 

060 

222 

105 

485 

054 

6523 

109 

110 
6424 
0191 A 

236 

475 

443 

334 

121 

464 

177 

369 

274 

275A 

287 

211 

316 

025 

450 


Incorrect  Calculation  of  Dat©  in  CPCI-28,  CPC  3 
Capability  to  Sp©cify  a  PUP  to  Edit  RCM  Not  Availabl© 

Non-display  of  Islands 

Need  for  Additional  Continuous  Tima  Laps©  Loop  Functionality 
Undocumented,  Non-standard  Implementation  of  "ADCCP  FLAG"  in 
Wideband  Interface 

RDA  Transmitter  Oil  Level  Sight  Gauge  Leaking  Oil 

Inconsistent  Degree  of  Accuracy  of  Displayed  Power  Measurements 

Missing  Hardware  on  Cable  Connector  in  RDA  Receiver  Cabinet  (UD4) 

Incorrect  Location  of  COMMON  Block  A317C4 

Inadequate  RDA  Data  Processor  (RDADP)  Small  Computer  System 

Interface  (SCSI)  Ribbon  Cable 

Inadequate  "CLUTTER  MAPS  HELP"  Screen 

Inconsistency  Between  Maximum  VIL  Value  and  Labeling  of  Color 
Category  Used  For  Its  Display 

Improper  Wiring  of  Transmitter  Cabinet  Blower  Assembly  B3 

Inconsistency  Between  CPCI-01  C5  and  Module  A10698 _ CSU 

K  PED  POSIT 


Incorrect  Plug  on  RAMTEK  4660  Diagnostics  Cable 

Need  for  Increased  Vertical  Resolution  for  Analyzing  Echoes  at  Distant 

Ranges 

Pedestal  Azimuth  Assembly  Leaking  Oil 

Pedestal  Elevation  Assembly  Leaking  Lubricate 

Need  for  Linear  Motion  Estimates  of  Echo  Features 

Proposed  Merger  of  Storm  Structure  (SS)  and  Storm  Track  Information 

(STI)  Alphanumeric  Products 

Industrial  Waste  Disposal  Container  at  Generator  Shelter  Not  Available 
PUP  Workstation  Audio  Alarm  (UD46)  Front  Panel  Potentiometer  Came 
Loose  and  Bent  Easily 

FMH-1 1  Part  D,  Chapter  2  and  3  (DRAFT)  Inadequately  Structured 
Use  of  Nondescriptive  Variable  Names  in  CPCI-04 
Graphic  Timeout  Occurs  When  Requesting  a  Hardcopy  While  Color 
Printer  is  Processing  a  Previous  Request 

Concurrent  3212  Power  Supply  Check  Procedures  Contained 
Unnecessary  Steps 

Inappropriate  Response  to  UCP  "RETURN  TO  PREVIOUS  MENU" 
(Function  Key  F2)  and  Response  Inconsistent  With  PUP 
"HARDCOPY  STOP"  Function  Not  Available 
Tolerances  on  P5  and  P5U  Voltage  Appear  to  be  Too  Critical 
RPG  Load  Shed  Messages  are  Not  Stored  at  the  PUP 
Inconsistent  PMI  Requirements/Procedures  for  the  RDA  Halon  Fire 
Suppression  System  Cylinder  (UD1A8) 

Inadequate  Configuration  Information  on  PUP  Color  Graphics  Printer 
(UD47) 


CPCI-28  Documentation  Problems 

Graphic  Tablet's  Protective  Plastic  Sheet  Unsecure 

PUP  User’s  Manual  Documentation  of  RPG  Directory  Mnemonics 

Incorrect 
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List  of  Prioritized  Category  II  Service  Reports 

• 

Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B)  1 

Rank 

SR  # 

Title 

546 

520 

Inadequate  "PRODUCT  FORWARD"  and  "PRODUCT  BACK” 

Functionality  for  Paired  Alphanumeric  (A/N)  Products 

547 

436 

Inconsistent  Acronym  for  Standby  on  RDA  Maintenance  Terminal 

548 

085 

Vertical  Lines  Occasionally  Shown  on  Graphic  Displays  When 

Replaying  Time  Lapse 

549 

116 

Maintenance  Control  Console  Not  Compatible  with  RDA  Cable 

■ 

550 

089 

Rear  Mounting  Holes  for  Small  Computer  System  Interface  (SCSI)  Fan 

Assembly  Misaligned 

551 

e524 

Need  for  Capability  of  PUP  to  Calculate  and  Display  Shear  Between 

Operator  Specified  Points  on  Velocity  Display 

552 

179 

Inability  to  Automatically  Display  One-Time  Request  Products  in 

Quarter  Screen  Mode 

553 

020 

Inadequate  Help  Screen  for  Free  Text  Message  Generation 

554 

171 

Inadequate  Small  Computer  System  Interface  (SCSI)  Drive  Assembly 

Bus  Cable  Clamp 

555 

235 

Documentation  for  Cleaning/Replacing  Generator  Shelter  Air  Filters  and 

Associated  Functional  Louvers  Not  Available 

556 

106 

Rear  Receiver  Hinged  Component  Rack  Interferes  with  Receiver 

Cabinet  Maintenance  Activities 

557 

037 

Ladder  Required  in  RDA  Shelter 

558 

513 

Error  Messages  Associated  with  Dial-up  Request  For  Products  with 

RADAR  SITES,  CITY,  or  COUNTY  NAMES  Maps 

• 

559 

e091 

Need  for  Capability  to  Position  Graphic  Tablet  Anywhere  on  PUP  Table 

560 

e024 

Need  foi  Cabinet  Lighting 

561 

114 

Radome  Heater  AC  Power  Cable  Will  Not  Remain  Connected  to 

Output  Power  Box 

562 

©115 

Need  For  Radome  Heater  Thermostat  Control 

563 

e275 

Need  for  Improved  Backspace  and  Scrolling  Capability  at  PUP 

Applications  Terminal 

564 

1 72A 

Entry  of  Storm  Motion  Parameters 

565 

074 

"PRODUCTS  IN  PUP  DATABASE"  Screen  Does  Not  Indicate  How  to 

Display  or  Delete  a  Listed  Product 

566 

e241  A 

Inconsistent  Use  of  Background  Map  Colors 

567 

465 

Rain  Water  Leaking  into  Generator  Shelter  (UD10) 

568 

175 

Response  to  "RETURN"  Key  When  Editing  UCP  "SEND  MESSAGE" 

Undocumented  and  Inconsistent  With  PUP 

569 

e219A 

Display  of  Maxhum  Value  Location(s) 

570 

el  81 

Need  for  Time  Lapse  to  Display  Original  Product  Resolution  at  Display 

Rates  Below  One  Frame  Per  Second 

■* 

571 

075 

Glare  on  the  Graphic  Tablet  Protective  Plastic  Sheet 

572 

108 

Inadequate  Documentation  on  Type  and  Size  of  Transmitter  Focus  Coil 

Air  Filter 

• 

573 

11  i 

Radome  Cooling  Fan  Inlet  Air  Louvers  Failed  to  Close  When  Fan  is 

Off 

574 

063 

Airport  Locations  on  "AIRPORT"  Background  Map  Changes  at  Different 

Magnifications 

575 

e409 

Need  for  Hodograph  Product  Produced  from  VAD  Winds 

576 

e321 

Need  for  PUP  to  Identify  Last  Selected  Routine  Set  (RPS)  List 

• 

577 

512 

Misalignment  of  LFM  Grid  and  RCM  Intermediate  Graphic  Reflectivity 

Blocks 
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List  of  Prioritized  Category  II  Service  Reports 

Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  I0T&E(1B) 

• 

Rank 

SR  # 

Title 

578 

372 

Preventive  Maintenance  Inspection  (PMI)  Procedures  for  PUP  Audible 
Alert  Operational  Check  Not  Available 

579 

e318 

Need  for  Matting  on  Interior  Floor  of  All  Radomes 

% 

580 

205 

"ARCHIVE  MENU  HELP  SCREEN"  Incorrectly  References  Megatape 

581 

6007 

Need  for  Imoroved  Metering  Circuitry  in  RDA  Transmitter  (Tx) 

582 

100 

Discrepancy  Between  Concurrent  Small  Computer  System  Interface 
(SCSI)  Commercial  Vendor  Manual  and  PTM,  Chapter  5 

- 

583 

306 

Relative  Spacing  of  Product  Annotation  Text  Characters  and  Special 
Symbols  Altered  After  Magnification 

584 

041 

Environmental  Winds  Edit  Screen  Data  Format  Inconsistent  With 
Operational  Upper  Air  Data  Format 

585 

e227A 

Need  for  Method  to  Select  and  Examine  Titles  of  Adaptation  Data 
Routine  Product  Set  (RPS)  Lists 

586 

428 

Inadequate  Separation  Between  Intermediate  Graphic  Display  and 
Function  Selection  Areas  on  RCM  Editing  Screen 

587 

051 

Optical  Disk  Drive  Cartridge  Ejection  Too  Forceful 

588 

160A 

Incorrect  Features  on  Velocity  Azimuth  Display  (VAD)  Products 

589 

6470 

Need  to  Program  Function  Keys  for  Concurrent  3212  and  3280 
Diagnostic  Procedures 

590 

341 

CPCI-03/CPCI-04  Shared  Module  Version  Discrepancy 

591 

305 

PUP  Occasionally  Displays  Incorrect  Background  Map  Resolution 

592 

339 

Inaccurate  Documentation  of  Adaptation  Data  Category  11,  RPG 
Directory  in  PUP 

• 

593 

213 

Page  Backward  Command  ("PAGB")  Incorrectly  Pages  Forward 

594 

379 

Transmitter  Key  Difficult  to  Remove 

595 

486 

Undocumented,  Non-Standard  Bit  Order  for  Raster  Data  Format 

596 

188 

Small  Computer  System  Interface  (SCSI)  Drive  Assembly  Rack  Mount 
Guide  Not  Aligned 

597 

099B 

Extraneous  Radials  of  Data  Shortly  After  Sunrise  and  Shortly  Before 
Sunset 

598 

el  61 

Need  for  Improved  Indication  at  PUP  Applications  Terminal  When 
"USER  FUNCTION  MENU  EDIT  Mode  is  Active 

599 

e270 

Need  for  Method  to  Display  in  Color  the  Three  Base  Products  at  RDA 

600 

247 

Inadequate  Documentation  of  Automated  Alert  Notification  Criteria 

601 

051 A 

Wavering  Data  on  RDA  Monitor 

602 

469 

Hardcopy  Capability  Not  Available  When  Running  Ramtek  Diagnostics 

603 

299 

Inconsistent  Documentation  of  P  JP's  Test  Pattern  #10 

604 

6021 B 

Need  for  Streamlined  Procedures  for  Displaying  the  List  of  Available 
RPG  Products 

605 

6364 

Need  for  Automatic  Selection  of  Clear- Air  Mode 

606 

e456 

Need  for  Minimum  Threshold  Value  Displayed  on  Contour  Products 

607 

143A 

Latitude/Longitude  Units 

608 

058 

"BACKGROUND  MAPS  FOR  PRODUCTS"  Help  Screen  at  the  Unit 
Control  Position  (UCP)  Not  Available 

' 

609 

e234 

Need  for  Water  Supply  at  RDA  Site 

610 

330 

"ARCHIVE  MENU  HELP  SCREEN"  Incorrect  for  Archive  Background 
Maps  Function 

611 

e046 

Need  for  Display  of  Products  in  Quarter  Screen  Mode  Using  Graphic 
Auto  Display  Mode 
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• 

List  of  Prioritized  Category  II  Service  Reports 
Opened  during  IOT&E(2)  or  Revalidated  from  IOT&E(1A)  and  IOT&E(1B) 


Rank 

SR  # 

Title 

612 

398 

Inconsistency  Between  VAD  Product  Times  and  Times  Displayed 
Below  Profiles  Within  the  Product 

613 

e216 

Need  for  Consistency  Between  UCP  Archive  Menu  Options  ard  Draft 
Federal  Meteorological  Handbook  #11  (FMH  #11) 

614 

&423 

Need  for  Capability  to  Filter  and  Blink  Data  Levels  of  Product  Overlays 

615 

188A 

Need  for  Improved  Graphics  Display  Editing 

616 

el  74 

Need  for  Consint  Response  to  PUP  and  UCP  Page  Commands 

617 

el  98 

Need  for  Capability  to  Define  Default  Time  Lapse  Execution  As  Either 
Continuous  Loop  or  One-Time  Display 

618 

e425 

Need  for  Overlay  Product  to  Display  Numeric  Values  of  Rain  Gauge 
Data 

619 

455 

ALERT  STATUS  Screen  Did  Not  Indicate  How  to  Cancel  Alerts 

620 

e242 

Need  for  Product.  Overlay,  and  Map  Mnemonics  to  be  Added  to 
Graphic  Tablet 

621 

133A 

Nonmeteorological  Azimuth  Values 

622 

131 

RDA  Applications  Terminal  Monitor  Has  a  Wavy  Presentation 

623 

282 

"WARNING  AREA"  Map  Not  Available 

624 

119 

Nonstandard  Depiction  of  5  Knot  Wind  Barb  on  VAD  Product 

625 

e233 

Need  for  RDA  Toi  et  Facility 

626 

e350 

Need  for  Automatic  Update  Option  for  Status  Screens 

627 

157 

FMH-11:  Inappropriate  Requirement  For  "RANGE  RING"  Map  to  be 
Associated  With  Archive  III  or  Non-Associated  PUP  Products 

628 

044 

Electromagnetic  Interference  (EMI)  Gasket  Loose  on  Transmitter  (Tx) 
Doors 

629 

e359 

Need  for  Capability  to  Horizontally  Magnify  Cross-Section  Products 

630 

322 

Julian  Date  Conversion  Incorrect  After  Year  1999 

631 

e365 

Need  for  Higher  Detailed  POLAR  GRID  Map  at  Four  and  Eight  Power 
Magnification 

632 

el  46 

Need  for  Reorientation  of  Vent  Thermostats 

633 

e039 

Need  for  Capability  to  Extrapolate  the  Highest  Entered  Environmental 
Wind  Value  Upwards  to  70,000  feet 

634 

433 

Missing  County  Boundaries  on  COUNTY  Background  Map  in 
Northwestern  Arkansas 

635 

142A 

Misnamed  Precipitation  Mode 

636 

el  52 

FMH-1 1:  Need  for  Reconsideration  of  Reflectivity  Categories  for  Data 
Near  Noise 

637 

e024B 

Need  for  Relocation  of  Range  Folding  (RF)  Color  Scale  Bar 

638 

e276 

Need  for  Maps/Overiays  Display  Toggle  Capability 

639 

el  82 

Need  for  Product  Names  on  'OVERLAY  ASSOCIATIONS  EDIT 
SCREEN" 

640 

e385 

Need  for  Echo  Top  Information  in  the  Radar  Coded  Message  (RCM) 
Intermediate  Graphic  Product 

641 

e345 

Need  to  Relocate  the  "BACKGROUND  MAP  VERSION"  Command  to 
the  "ADAPTATION  DATA  MENU" 

642 

e389 

Need  for  the  Ability  to  Independently  Control  Transmitter  Intake  and 
Exhaust  Dampers 

643 

e086 

Need  for  "Modify  Line"  Capability  on  "EXAMINE/EDIT  USER 
FUNCTION"  Edit  Screen 
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APPENDIX  D  -  ADDITIONAL  RELIABILITY  DATA 


1.  RDA:  The  RDA  d  .nonstrated  the  lowest  reliability  (MTBM  (total  corrective)  of  ^  1 
hours'  of  the  three  functional  areas  and  had  the  greatest  impact  on  maintenance  workload. 

a.  Of  the  29  corrective  maintenance  events  required  at  the  RDA  dunng  Part  B.  26 
were  for  inherent  malfunctions. 

(1)  Eleven  of  the  inherent  RDA  malfunctions  were  transmitter  problems.  On  six 
of  these  occasions,  the  preproduction  transmitter  was  inoperative,  indicating  fault  alarms 
such  as  "Focus  Coil  Failure"  or  "Mod  Switch  Failure"  and  system  operation  was  restored 
after  resetting  the  transmitter  fault  panel  alarms;  two  of  these  occurred  after  a  power 
interruption  at  the  RDA  site.  Three  actions  involved  hardware  or  LRU  replacement:  (1) 
replacement  of  a  charging  switch  module,  (2)  replacement  of  a  trigger  amplifier  module, 
and  (3)  replacement  of  two  transmitter  blower  fuses.  One  action  involved  adjustment  of 
the  pulse  forming  network  PFN  voltage  to  alleviate  a  high  transmitter  pea1-  power  condition. 
The  11th  action  involved  extensive  transmitter  alignments  and  Klystron  tuning  required  to 
correct  a  high  delta  system  calibration  indication  associated  with  a  decrease  in  transmitter 
output  power.  The  test  team  also  had  19  transmitter  maintenance  actions  during  Part  A 
of  the  test. 

(2)  Nine  of  the  inherent  RDA  malfunctions  were  corrected  by  reinitializing  the  RDA 
software.  Four  were  for  corrective  actions  associated  with  clearing  alarms  (e.g..  "Lin 
Channel  Cal  Constant  Degraded,"  "Lin  Channel  Cal  Check  Indicates  Maintenance 
Required,"  and  "Radial  Time  Interval  Error)."  Two  of  the  software  reinitializations  were 
required  because  the  RDA  maintenance  terminal  locked  up  and  would  not  respond.  Three 
of  the  reinitializations  were  required  when  the  applications  program  stopped  running;  two 
of  these  occurred  after  a  power  interruption  at  the  RDA  site. 

(3)  Four  other  RDA  maintenance  actions  involved  LRU  or  hardware  replacement. 
The  antenna  power  monitor  required  replacement  twice.  The  other  two  actions  involved 
replacement  of  a  power  supply  in  the  pedestal  control  unit  and  replacement  of  the  filter 
in  the  transmitter  air  intake  duct. 

(4)  One  inherent  RDA  action  required  adjustment  of  me  backup  generator  transfer 
delay  time  because  the  HDA  site  failed  to  transfer  to  backup  power. 

(5)  The  last  inherent  RDA  malfunction  occurred  when  an  "Elevation  Gearbox  Oil 
Level  Low"  message  occurred.  Several  cables  and  LRUs  were  replaced;  however,  the 
exact  cause  of  the  failure  was  never  determined. 

b.  The  one  induced  RDA  maintenance  action  was  associated  with  a  defective  post¬ 
charge  regulator  which  occurred  while  replacing  the  defective  charging  switch  module. 

c.  The  two  no-oefect  RDA  maintenai.ee  actions  were  attributed  to  RDA  alarms  which 
cleared  before  maintenance  could  respond. 

2.  RPG:  The  RPG  demonstrated  the  second  lowest  reliability  (MTBM  (total  corrective) 
of  78.6  hours)  of  the  functional  areas. 

a.  Of  the  23  corrective  maintenance  actions  required  at  the  RPG  during  Part  B.  20 
were  for  inherent  malfunctions. 

(1)  The  most  significant  problem  was  the  failure  of  the  RPG  to  recover 
automatically  following  power  transitions;  this  occurred  12  times.  Ten  times  system 
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operation  was  restored  through  a  reset/restart  of  the  RPG  software.  However,  one  outage 
required  reloading  the  RPG  software  to  correct  the  problem,  and  the  other  outage  required 
a  reconfiguration  of  the  software  interfaces. 

(2)  Seven  other  inherent  RPG  corrective  maintenance  actions  required  only  a 
reset/restart  of  the  RPG  software  to  correct  the  problem.  Three  were  associated  with  a 
disruption  of  narrowband  communications,  two  were  associated  with  wideband 
communications  problems  causing  RPG  discontinuity/loadsnedding  messages  or  an 
unsolicited  RDA  disconnect,  one  was  required  to  correct  an  Archive  III  problem,  and  one 
was  required  to  restore  operations  when  the  RPG  went  down  for  unknown  reasons. 

(3)  The  two  remaining  inherent  RPG  malfunctions  involved  inoperative  monitors 
at  the  unit  control  position  (UCP).  One  monitor  had  to  be  replaced,  and  one  was  powered 
off/on  to  restore  operations. 

b.  The  remaining  three  RPG  maintenance  events  were  no-defect  maintenance  events. 
All  three  were  "cannot  duplicate"  events.  One  involved  a  “RDA/RPG  Communication  Link 
Broken"  message,  but  the  system  recovered  automatically  before  maintenance  responded. 
One  involved  an  Archive  III  problem  that  was  no  longer  evident  when  maintenance  arrived. 
The  remaining  event  involved  narrowband  line  noise  which  maintenance  personnel  were 
unable  to  duplicate. 

3.  PUP:  The  PUP-demonstrated  MTBM  {total  corrective)  was  125.6  hours. 

a.  Of  the  35  corrective  maintenance  actions  required  at  the  three  operational  PUP 
sites  during  Part  B,  27  were  for  inherent  malfunctions. 

(1 )  The  most  significant  problem  was  graphics  lockups  associated  with  the  Ramtek 
graphics  processor.  Nine  of  the  ten  inherent  malfunctions  associated  with  the  graphic 
processor  were  corrected  by  reseating  the  hard  cursor  card:  one  of  these  occurred  after 
a  power  interruption.  The  10th  inherent  malfunction  was  corrected  through  replacement 
of  the  PUP  AC  line  filter  and  the  replacement  of  the  graphics  interface  card  in  both  the 
Ramtek  graphics  processor  and  the  Concurrent  3212  processor. 

(2)  The  Archive  IV  optical  disk  drive  unit  required  four  maintenance  actions  due 
to  inherent  malfunctions.  Three  required  adjusting  or  tightening  the  disk  ejection  lever 
because  an  optical  disk  was  stuck  or  the  drive  would  not  activate.  The  fourth  required 
removal  and  replacement  of  a  failed  optical  disk  drive  unit. 

(3)  There  were  five  additional  maintenance  actions  required  to  replace  failed 
LRUs.  The  LRUs  replaced  were  the  Concurrent  3212  processor  multiple  peripheral 
controller,  a  color  monitor,  an  applications  terminal  monitor,  a  color  printer,  and  a  1/4  inch 
streaming  tape  drive. 

(4)  Of  the  eight  remaining  maintenance  actions  associated  with  inherent  PUP 
malfunctions,  six  involved  conditions  where  the  PUP  was  inoperable  and  system  operation 
was  restored  by  reinitializing  the  PUP  software.  One  action  involved  reattaching  terminals 
on  wires  in  the  PUP  cabinet,  and  the  final  action  required  powering  a  monitor  off/on  to 
correct  a  system  console  blank  screen. 

b.  Of  the  seven  corrective  maintenance  actions  required  for  induced  PUP 
malfunctions,  two  were  for  switches  or  circuit  breakers  which  were  not  set  to  the  "on" 
position  during  power-up  procedures.  Two  other  actions  involved  the  color  printer:  one 
for  an  incorrectly  positioned  media  mode  selector,  and  one  when  the  print  carriage  was 
not  properly  locked  in  place.  One  action  was  the  replacement  of  an  applications  terminal 
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monitor  which  had  not  had  a  previous  problem  corrected  before  being  reinstalled.  One 
action  was  required  to  correct  a  display  foldover  problem  on  a  recently  replaced  color 
monitor.  The  final  corrective  action  for  an  induced  PUP  malfunction  was  attributed  to  a 
faulty  optical  disk. 

c.  There  was  only  one  no-defect  PUP  maintenance  action.  This  involved  an  optical 
disk  error  message  which  maintenance  personnel  were  unable  to  duplicate. 
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APPENDIX  E  -  GLOSSARY 


Archive  I 

Archive  II 

Archive  III 

Archive  IV 

Assess 

Associated  User 

Base  Products 

Capability 

Central  Processing  Unit 
Critical  Success  Index 
Dealiasing 

Derived  Products 

Did  Not  Meet  Requirement 


The  capability  to  store  and  retrieve  analog  time-domain  data 
output  from  the  receiver. 

The  capability  to  store  and  retrieve  digital  base  data  and  status 
information  output  from  the  signal  processor. 

The  capability  to  store  and  retrieve  selected  NEXRAD  products 
and  status  information  from  the  RPG.  Archive  III  data  may  be 
read  by  Archive  IV. 

The  capability  to  store  and  retrieve  selected  NEXRAD  products, 
status  information,  and  background  maps  from  the  PUP.  The 
PUP  training  mode  makes  use  of  this  capability. 

Used  to  provide  information  about  system  capabilities  without 
assigning  ratings.  This  term  applies  when  user  requirements 
are  not  available  or  may  not  be  appropriate  for  the  phase  of 
development;  however,  information  is  needed  to  support  the 
user  or  the  decision-making  process. 

A  PUP  that  is  connected  to  an  RPG  using  a  dedicated 
communications  line.  Products  and  status  information  is 
automatically  sent  from  the  RPG  to  the  PUP. 

Those  products  that  represent  fields  of  the  three  moments 
directly  measured  by  NEXRAD  (i.e.,  reflectivity,  mean  radial 
velocity,  and  spectrum  width). 

The  percentage  of  DOD  warnings  that  are  both  correct  and 
provide  the  desired  lead  time. 

That  part  of  the  computer  that  interprets  and  executes 
instructions. 

An  index,  used  only  by  DOC.  that  is  a  measure  of  a 
forecaster's  ability  to  forecast  effectively  and  correctly. 

The  process  of  assigning  the  correct  velocity  to  Doppler-derived 
wind  data.  Wind  velocities  are  determined  by  (Doppler)  shifts 
in  the  received  signal  frequency  from  that  transmitted.  Data 
with  incorrect  velocities  assigned  are  the  result  of  not  sampling 
at  a  high  enough  rate  to  determine  the  exact  Doppler  shift  of 
the  received  frequency. 

Those  products  generated  within  the  RPG  that  represent  either 
some  combination  of  base  products  or  a  base  product  that  has 
been  enhanced  or  otherwise  changed  by  the  use  of  automatic 
processing  techniques. 

Level  of  performance  was  below  the  users’  stated  requirement. 
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GLOSSARY  (continued) 


Evaluate 


False  Alarm  Rate 
Met  Requirements 
Narrowband 

Nonassociated  User 

Nowcast 


Used  to  determine  a  system’s  ability  to  meet  the  users'  stated 
requirements.  Quantitative  or  qualitative  methods  of  evaluation 
may  be  used.  Ratings  of  "met"  or  "did  not  meet  requirements" 
will  be  assigned. 

The  percentage  of  incorrect  warnings  issued. 

Performance  met  or  exceeded  the  users'  stated  requirements. 

The  communications  link  between  the  RPG  and  PUP  that 
transmits  NEXRAD  products  and  status  information  via 
telephone  lines. 

A  PUP  that  is  connected  to  an  RPG  using  dialup  commu¬ 
nications.  Products  must  be  individually  requested  from  the 
RPG. 

A  combination  of  reports  of  current  weather  conditions  in  the 
local  area  and  a  short-term  forecast  for  3  to  6  hours. 


Other  Users 


PFI 


Probability  of 
Detection 

Principal  User 


PUES 


Range  Folding 


Other  users  of  the  NEXRAD  system  include  federal  government 
agencies  other  than  the  principal  users;  state  and  local 
government  agencies;  and  private  sector  users  such  as  airline 
companies,  consulting  meteorologists,  news  media,  and 
universities. 

Primary  Fault  Isolation  (PFI)  is  a  method  of  troubleshooting 
that  makes  use  of  PTM  fault  isolation  flow  charts,  built-in  test, 
indicators,  displays,  printed  listings,  and  self-diagnostic  internal 
logic,  either  as  loadable  diagnostic  software  or  firmware. 

The  percentage  of  confirmed  weather  events  covered  by  cor¬ 
rect  warnings. 

Operationally  oriented  agencies  within  DOC,  DOD,  and  DOT 
which  use  weather  radar  information  to  perform  or  support 
their  activities. 

A  Principal  User  External  System  (PUES)  is  an  existing  or 
planned  principal  user  information  network  or  other  automated 
system  with  which  one  or  more  NEXRAD  units  must  interface. 
PUES  may  interface  with  NEXRAD  through  either  the  RPG  or 
PUP. 

The  placement  of  a  single  weather  feature  at  multiple  ranges 
from  the  radar.  In  order  to  resolve  a  large  span  of  velocities, 
the  radar  must  transmit  at  a  high  pulse  repetition  rate.  Range 
folding  occurs  because  the  radar  cannot  determine  if  a  returned 
signal  was  caused  by  the  most  recently  transmitted  pulse  or 
earlier  transmitted  pulses. 
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GLOSSARY  (continued) 


Volume  Scan 


Wideband 


The  continuous  rotation  of  the  antenna  in  azimuth  while 
automatically  adjusting  the  antenna  elevation  in  discrete  steps. 
During  IOT&E(2),  a  volume  scan  took  from  5  to  10  minutes 
depending  on  which  volume  scan  was  selected  by  the  operator. 

The  communications  link  between  the  RDA  and  RPG  functional 
area  that  transmits  NEXRAD  base  data,  status  information,  and 
RDA  control  commands  via  fiber  optics,  microwave,  or  other 
communications  media  with  a  capacity  greater  than  a  telephone 
line  (narrowband). 


E-3 


APPENDIX  F  -  SELECTED  OPERATOR  QUESTIONNAIRE  QUESTIONS 


Primary  MOE  Questions  that  have  Criteria. 


Question 

Number 

Primary 

MOE 

Question 

1 

E-1-1 

What  was  the  overall  effectiveness  of  NEXRAD  as  an  aid  for  you 
in  preparing  accurate  and  timely  weather  warnings? 

2 

E-2-1 

What  was  the  impact  on  workload  when  you  used  only  the 
NEXRAD  PUP  to  perform  existing  agency  requirements? 

3 

E-2-2 

What  was  the  impact  on  workload  when  you  used  the  NEXRAD 
PUP  and  UCP  to  perform  existing  agency  requirements? 

4 

E-4-1 

What  was  the  overall  effectiveness  of  NEXRAD  in  providing 
requested  products  in  a  timely  manner  when  you  operated  the  unit 
in  various  weather  scenarios  at  the  representative  maximum  load? 
(Sterling,  VA  configuration) 

5 

E-5-1 

What  was  the  overall  effectiveness  of  NEXRAD  as  an  aid  for  you 
in  preparing  weather  advisories? 

6 

E-6-1 

What  was  the  overall  effectiveness  of  NEXRAD  as  an  aid  for  you 
in  preparing  short-range  forecasts?  (0-6  hrs) 

7 

E-6-2 

What  was  the  overall  effectiveness  of  NEXRAD  as  an  aid  for  you 
in  taking  surface  weather  observations? 

8 

E-6-3 

What  was  the  overall  effectiveness  of  NEXRAD  as  an  aid  for  you 
in  preparing  and  presenting  weather  briefings? 

9 

E-6-4 

What  was  the  effectiveness  of  NEXRAD  as  an  aid  for  you  in 
briefing  traffic  management  on  weather  problems  that  could  impact 
local  traffic  flow  or  local  air  traffic  control  capabilities? 
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